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[57] ABSTRACT 

A small, portable, hand-held electronic personal organizer 
performs voice recognition on words spoken by a user to 
input data into the organizer and records voice messages 
from the user. The spoken words and the voice messages are 
input via a microphone. The voice messages are compressed 
before being converted into digital signals for storage. The 
stored digital voice messages are reconverted into analog 
signals and then expanded for reproduction using a speaker. 
The organizer is capable of a number of a different functions, 
including voice training, memo record, reminder, manual 
reminder, timer setting, message review, waiting message, 
calendar, phone group select, number retrieval, add phone 
number, security, and "no" logic. During such various func- 
tions, data is principally entered by voice and occasionally 
through use of a limited keypad, and voice recordings are 
made and played back as appropriate. A visual display 
provides feedback to the user. During the various function, 
the user can edit various different data within the organizer 
by eliminating or correcting such data or entering new data, 

6 Claims, 19 Drawing Sheets 
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VOICE ACTIVATED PERSONAL 
ORGANIZER 



BACKGROUND OF THE INVENTION 

1 . Field of the Invention 

The present invention relates to personal organizers for 
electronically storing messages, reminders, phone numbers, 
addresses, and other such data, and more particularly to 
personal organizers which are voice activated. 

2. History of the Prior Art 

Many types of electronic personal organizers or "data 
banks" are presently available. Such organizers range from 
simple devices that allow for storage of phone numbers, 
addresses and appointments, to more complicated devices 
that approach the capabilities of small computers. All such 
devices require data to be entered using a keyboard. Simpler 
devices may use a calculator type keyboard, while more 
complex devices typically require a computer/typewriter 
type of keyboard. 

In presently available electronic personal organizers, the 
user typically selects a function by pressing one or more 
keys on the keyboard. The user then enters data using the 
keyboard, usually filling out a predefined form for the 
function selected. For example, a phone directory entry 
typically requires the user to type the name of the person or 
organization to be added to the directory, in a specific field. 
The user then indicates by keystroke when the filling of the 
field is finished. The organizer then automatically moves to 
the next field, where the user inputs the phone number. This 
field may be further subdivided into area code and phone 
number. When entry of the information is finished, the user 
indicates by keypress that the entry is to be saved. 

Retrieval of data is accomplished by similar keypress 
operations, in conventional electronic personal organizers. 
The user again begins by selecting the function, following 
which a search for the desired information is begun. In the 
case of the phone directory previously referred to, the user 
may simply scroll through the directory looking for the 
desired entry, with a single keypress being used to advance 
from one entry to the next. A more sophisticated search is 
often provided, by which the user may type the first letter (or 
perhaps more) of the name. This causes the directory to skip 
to the appropriate alphabetical region. 

In conventional electronic personal organizers, a second 
type of data is often stored. Instead of being stored for later 
retrieval at the user's initiation, this data is interpreted by the 
organizer so as to ultimately cause the organizer to take a 
particular action at a later time, with no further action on the 
part of the user being required. For example, a time and date 
can be entered in a reminder function. The organizer keeps 
track of the time in order to automatically alert the user when 
the selected time arrives, with no user intervention being 
required. Thus, an alarm function is performed. Typically, a 
message is associated with the alarm function to provide the 
reminder with some context. The message as well as the 
alarm time are entered by keystrokes in a form similar to that 
described previously in connection with the phone directory 
example. 

When using conventional electronic personal organizers 
in the manner described, the user must type in information 
using a small keypad. The keypad must be of minimum size 
in order for the keys to be usable. This conflicts with the 
need to make the organizer as small and portable as possible. 
Elimination of the need for a complete keypad for data input 
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and retrieval would eliminate the need for compromise, 
allowing the organizer to be made small and portable and at 
the same time easily used. As previously noted, conventional 
electronic personal organizers typically require a computer/ 
typewriter type keyboard for complete flexibility in entry of 
number data, such as phone numbers, times and dates, and 
text data, such as memos. This requires a certain level of 
skill on the user's part, and can be quite time consuming. 
Also, the large number of keys required results in the unit 
being relatively large. 

For this reason, voice activation and other voice recog- 
nition techniques have provided a useful alternative to the 
need for elaborate user interfacing through use of a large 
keyboard, in certain electronic devices. Examples of voice 
activated electronic devices include remote controls which 
utilize sophisticated electronics to recognize spoken words, 
translate the commands of the user into traditional digital 
remote control signals, and transmit the control signals to a 
controlled device. Examples of such systems are provided 
by co-pending application Ser. No. 07/915,112 of Bisson- 
nette et al., entitled Voice Operated Remote Control Device, 
by co-pending application Ser. No. 07/915,938 of Bisson- 
nette et al., entided Voice Recognition Apparatus and 
method, and by co-pending application Ser. No. 07/915,114 
of Fischer, entitled Remote Control Device. All three appli- 
cations were filed on Jul. 17, 1992 and are commonly 
assigned with the present application. 

A further example of a voice operated remote control 
system is provided by co-pending application Ser. No. 
08/1 13,394 of Fischer et al., entitled Voice Operated Remote 
Control System. The Fischer et al. application, which was 
filed Aug. 27, 1993 and which is commonly assigned with 
the present application, describes a system which includes a 
remote control device responsive to the voice commands of 
the user to transmit representations of the voice commands 
to a controlled device. The controlled device produces voice 
signals in response to the transmitted representations, and 
includes voice recognition circuitry for recognizing the 
transmitted voice commands and executing action routines 
denoted thereby. 

Voice recognition techniques have also been applied to 
systems capable of performing organizer type functions. 
Typically, such systems are very large in size in order to 
accommodate the data storage and other functions. This 
severely limits their applicability to small, portable, hand- 
held applications. An example of such systems is provided 
by U.S. Pat. No. 5,014,317 of Kita et al., which describes 
recording and reproducing apparatus in which externally 
input voice commands stating an alarm time are converted 
into voice data for storage in a memory together with an 
associated message. When the alarm time is reached, the 
corresponding voice data stored in the memory is read out 
and audibly reproduced so as to sound the alarm time and 
play back the associated message. 

The system described in Kita et al. is exemplary of 
extremely complex systems which are difficult and expen- 
sive to implement, and yet which are limited in terms of their 
flexibility in changing or correcting data and in terms of the 
functions which they otherwise can perform. Such systems 
typically carry out voice recognition and voice recording 
simultaneously, thereby requiring a substantial amount of 
hardware. 

In addition to the large, elaborate, computer type systems 
such as that described in Kita et al., voice recognition 
techniques have been applied to smaller systems where the 
functions may be simpler and easier to perform in compact 
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environments. An example of this is provided by U.S. Pat 
No. 4,882,685 of van der Lely, which patent describes a 
calculator responsive to certain action words such as "add", 
"subtract", "multiply", and "divide". Other examples of 
such systems are. provided by patents relating to automatic 
telephone dialers. Such patents include U.S. Pat. No. 4,644, 
107 of Clowes et ah, U.S. Pat. No. 5,007,081 of SchxnuckaJ 
et al., U.S. Pat No. 4,928,302 of Kaneuchi et al., and U.S. 
Pat. No. 4,864,622 of Iida et al. 

In developing electronic personal organizers, it has 
become apparent that digital voice recording is a signifi- 
cantly easier and more natural method than text to input and 
store data. Furthermore, input data in the form of numbers, 
dates, times, and the like, can be handled in a more natural 
and simpler way by utilizing voice recognition technology. 
However, while such techniques greatly simplify use of the 
organizer, they do so at the expense of considerably greater 
complexity in the implementation of the organizer. This is a 
particular problem if the organizer is to be produced in a 
small, portable form. Thus, whereas a text memo typed into 
a conventional organizer using a keyboard will typically 
require 7 or 8 bits for each character, and a simple message 
such as "Call the office and speak with Bob" will typically 
require 238-272 bits, plus several additional "overhead" bits 
to keep the stored information organized, an organizer 
utilizing digital voice recording and voice recognition of 
data input will typically require 16,000-32,000 bits for 
proper storage of a sentence requiring only 1-2 seconds to 
speak. In addition to such storage requirements, there is the 
added requirement of providing the electronics for voice 
input and playback, including a microphone, a speaker, and 
appropriate amplifiers. 

In such organizers utilizing voice recognition, there is the 
increased requirement of additional componentry for imple- 
menting the voice recognition process. There is also the need 
for sufficient processing power to enable a voice recognition 
algorithm to be run, and increased memory requirements 
both for the program memory, typically a ROM, for storage 
of the recognition algorithm and other parts of the organizer 
program, as well as a read-write type memory, typically a 
RAM, for storage of information pertaining to the voice of 
the user. At that, such added memory requirements are but 
a small fraction of the memory requirements for voice 
recording. 

To make the implementation practical, the voice recog- 
nition requirements must be limited. The method of use 
should provide context for the recognition function, to allow 
for voice recognition with the limited processing power 
obtainable in a small, portable device. For example, requir- 
ing recognition during a continuous stream of speech on a 
few key words placed at varying points within the utterance 
would require a large, complex computer system. 

Accordingly, there is a need for an electronic personal 
organizer capable of digitally storing reasonable amounts of 55 
message data, and facilitating the use thereof through appro- 
priate voice recognition techniques. At the same time, such 
an organizer must be capable of implementation in a small, 
portable, hand-held package in order for it to be practical 
and to lend itself to large-scale use. 
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BRIEF SUMMARY OF THE INVENTION 

Briefly stated, the present invention provides an electronic 
personal organizer which provides for data entry and 
retrieval using voice for commands as well as data input. 
Two types of voice interaction are provided; digital voice 
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recording and voice recognition. A minimal number of 
buttons or other manual controls are required, enabling a less 
complicated device with ease of user interface. 

In electronic organizers according to the invention, voice 
recognition is performed on words spoken by the user to 
input data into the organizer. At the same time, voice 
messages from the user are recorded in the organizer. The 
organizer follows a set routine so that it can readily be 
determined when voice input from the user comprises input 
data for the voice recognition process and when the voice 
input is a message to be stored. The voice messages are 
preferably compressed and then converted into digital sig- 
nals for storage in a memory. The spoken words and the 
voice messages may be input using a microphone. 

In electronic organizers according to the invention, voice 
recognition is carried out by implementing a voice recog- 
nition algorithm in conjunction with templates previously 
made from a user's voice and stored. When setting up the 
organizer for use, the user is required to speak each of a 
limited vocabulary of key words into the organizer, for 
creation and storage of the digital templates corresponding 
to the user's spoken words. Thereafter, as the user speaks the 
various words, the spoken words are compared with the 
stored templates in search of matches which denote recog- 
nition of certain ones of the key words. The various tem- 
plates are trained until acceptable matches with the user's 
voice can be confirmed. Thereafter, the templates can be 
periodically corrected or retrained as appropriate. 

The voice messages stored in the electronic organizer are 
selectively played back by converting such messages into 
analog signals and amplifying and filtering such signals 
before application to a speaker to produce the audio sounds 
corresponding thereto. 

The electronic organizer includes a liquid crystal display 
or similar display together with a limited keypad. The 
keypad provides for manual entry of a limited number of 
selections and commands in connection with the voice 
recognition process. The display provides information feed- 
back to the user, to facilitate interaction between the user and 
the organizer. 

The electronic organizer includes a microcontroller hav- 
ing a plurality of different memories for storage of infor- 
mation together with a microprocessor and a stored pro- 
gram. The program establishes a set operating routine for the 
organizer, whereby various different predetermined func- 
tions may be carried out. By having a set operating routine, 
the organizer can determine which voice inputs require 
voice recognition in accordance with the limited vocabulary 
of key words and which voice inputs comprise voice mes- 
sages to be stored. 

Various functions which the electronic organizer is 
capable of performing include memo record, reminder, 
manual reminder, timer setting, message review, phone 
group select, number retrieval, add phone number, security, 
and "no" logic. 

BRIEF DESCRIPTION OF THE DRAWINGS 

A better understanding of the invention may be had by 
reference to the following specification in conjunction with 
the accompanying drawings, in which: 

FIG. 1 is a block diagram of a voice activated personal 
organizer in accordance with the invention; 

FIG. 2 is a plan view of the voice activated personal 
organizer electronically represented by the block diagram of 
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FIG. 1 and showing the limited keypad made possible in 
accordance with the invention; 

FIG. 3 is a pictorial representation of the different types 
of data stored in the DRAM of the voice activated personal 
organizer of FIG. 1; 

FIG. 4 is a flow diagram of the idle mode/select operation 
mode software routine implemented in the control program 
of the voice activated personal organizer of FIG. 1; 

FIG. 5 is a flow diagram of the set clock operation mode 
software routine implemented in the control program of the 
voice activated personal organizer of FIG. 1; 

FIG. 6 is a flow diagram of the voice training operation 
mode software routine implemented in the control program 
of the voice activated personal organizer of FIG. 1; 

FIG. 7 is a flow diagram of the memo record operation 
mode software routine implemented in the control program 
of the voice activated personal organizer of FIG. 1; 

FIG. 8 is a flow diagram of the reminder setting operation 
mode software routine implemented in the control program 
of the voice activated personal organizer of FIG. 1; 

FIG. 9 is a flow diagram of the manual reminder operation 
mode software routine implemented in the control program 
of the voice activated personal organizer of FIG. 1; 

FIG. 10 is a flow diagram of the timer setting operation 
mode software routine implemented in the control program 
of the voice activated personal organizer of FIG. 1; 

FIG. 11 is a flow diagram of the message review operation 
mode software routine implemented in the control program 
of the voice activated personal organizer of FIG. 1; 

FIG. 12 is a flow diagram of the waiting message opera- 
tion mode software routine implemented in the control 
program of the voice activated personal organizer of FIG. 1; 

FIG. 13 is a flow diagram of the calendar operation mode 
software routine implemented in the control program of the 
voice activated personal organizer of FIG. 1; 

FIG, 14 is a flow diagram of the phone group select 
operation mode software routine implemented in the control 
program of the voice activated personal organizer of FIG. 1; 

FIG. 15 is a flow diagram of the number retrieval opera- 
tion mode software routine implemented in the control 
program of the voice activated personal organizer of FIG. 1; 

FIG. 16 is a flow diagram of the add phone number 
operation mode software routine implemented in the control 
program of the voice activated personal organizer of FIG. 1; 

FIG. 17 is a flow diagram of the security operation mode 
software routine implemented in the control program of the 
voice activated personal organizer of FIG. 1; 

FIG. 18 is a flow diagram of the "No" logic operation 
mode software routine implemented in the control program 
of the voice activated personal organizer of FIG. 1; and 

FIGS. 19A-19G are illustrations of different visual dis- 
plays provided by the voice activated personal organizer of 
FIG. 1 during various operations thereof. 

DETAILED DESCRIPTION 

FIG, 1 is a block diagram of a voice activated personal 
organizer 10 in accordance with the invention. The descrip- 
tion of FIG. 1 and of the various flow diagrams in subse- 
quent figures of the drawings which relate thereto are 
provided by way of example only. Accordingly, it will be 
apparent to those skilled in the art that other arrangements 
and software routines are possible in accordance with the 
invention. 
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In the present example, the organizer 10 includes a 
microcontroller 12, which is the key component of the 
organizer 10 inasmuch as it manages operation of the overall 
system of the organizer 10 in addition to operating the voice 
recognition algorithms. The microcontroller 12 includes a 
ROM (read only memory) 14 which stores a program for 
operating the organizer 10 as well as static data used in 
implementing the functions of the organizer 10. The ROM 
14 is shown in FIG. 1 as an internal part of the microcon- 
troller 12, but it will be understood that the ROM 14 and 
other components like it can alternatively comprise separate 
components located external to the microcontroller 12. 

Also contained within the microcontroller 12 is a RAM 
(random access memory) 16 which is used for local tempo- 
rary storage of data necessary for the microcontroller 12 to 
fully implement the functions required. The microcontroller 
12 further includes an A/D (analog-to-digital) converter 18 
for converting inputted voice signals to digital form. The 
microcontroller 12 includes LCD (liquid crystal display) 
drive circuitry for driving an LCD 20. 

In addition to the RAM 16 which is internal to the 
microcontroller 12, the organizer 10 has a larger amount of 
RAM memory external to the microcontroller 12, for more 
permanent storage of several types of data. This is accom- 
plished by a DRAM 22, although it should be understood 
that other types of memories can also be used, As shown in 
dotted outline, additional expansion memories can be pro- 
vided as necessary. The DRAM 22 stores "voice templates" 
that are collected during the set-up process to enable rec- 
ognition of a specific user's voice. The DRAM 22 is .also 
used to store the dates and times for any reminders, as well 
as phone numbers for the phone directory function. The 
DRAM 22 also contains "flags" for each such item indicat- 
ing, for example, that a phone number is a home number or 
a work number, or that a reminder is to occur weekly, or 
daily, or one time only. The bulk of the DRAM 22 is used 
for the storage of digital voice recordings. 

In the organizer 10 of FIG. 1, a sound transducer for 
incoming voice commands and messages from the user is 
provided by a microphone 24. The microphone 24 converts 
the acoustic waves generated by the user's voice into analog 
electronic signals, which are amplified and filtered by an 
analog input amplifier and filter 26. The analog input ampli- 
fier and filter 26 amplifies and filters the signals from the 
microphone 24 in such a way as to optimize the capabilities 
of the voice recognition algorithms employed by the micro- 
controller 12. At the same time, such analog signal is also 
amplified and filtered by the analog input amplifier and filter 
26 in such a way as to optimize the recording quality. 
Consequently, the overall transfer function of the signal path 
from the microphone 24 to a voice compression and decom- 
pression circuit 28 is different from the transfer function of 
the path from the microphone 24 to the A/D converter 18 to 
implement the voice recognition algorithms. The difference 
is necessary because the optimal signals for the two pro- 
cesses, namely recording of messages and voice recognition, 
are different Such differences are in part made necessary by 
the hardware approach to compression and decompression 
provided by the voice compression and decompression 
circuit 28. 

As previously noted, the voice signal received by the 
microphone 24 and processed by the analog input amplifier 
and filter 26 is applied directly to the A/D converter 18 of the 
microcontroller 12 for voice recognition. The A/D converter 
18, which could be external to the microcontroller 12 instead 
of forming an internal component thereof as shown, con- 
verts the analog voice signal into a digital signal which the 
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microcontroller 12 can use. At the same time, voice signals 
to be recorded are provided by the analog input amplifier and 
filter 26 to the voice compression and decompression circuit 
28. In the present example, the voice compression and 
decompression circuit 28 implements a Continuously Vari- 5 
able Slope Delta Modulation (CVSD) compression and 
decompression algorithm. Consequently, the circuit 28 is a 
form of A/D converter, but at the same time one that 
significantly processes and thus compresses the amount of 
digital data that results for conversion of the analog voice 
signal to the digital voice signal. This allows a minimum of 
memory to be used for recording the voice messages. The 
data comprising such voice messages is stored in the DRAM 
22. The CVSD compression and decompression algorithm 
also converts the stored compressed digital voice signals 
back into analog voice signals for playback, via an analog 15 
output amplifier and filter 30 and a speaker 32. The analog 
output amplifier and filter 30 optimizes the sound quality for 
reproduction by the speaker 32. 

The LCD (liquid crystal display) 20 is utilized to visually 20 
feed back information to the user of the organizer 10. As 
shown in FIG. 2 as well as in FIG. 1, the organizer 10 has 
a keypad 34 of limited size, to enable the user to interact 
with the organizer 10. The keypad 34 has but 12 keys, which 
are denoted and used to perform functions as follows: 25 



Key 


Function 


record 


used for making voice recordings 


phone 


used for phone directory input and retrieval 


select 


used to select functions for review/use 


time 


used for voice input of times/dates and other data 


play 


used for playing back recordings 


next 


used to advance to the next item 


prev 


used to move to the previous item 


stop 


used to abort the present operation 


train 


used for training the organizer to the user's 




voice 


save 


used to store information in the RAM 


erase 


used to eliminate information from the RAM 


edit 


used for entering editing and manual input modes 



As shown in FIG. 1, the organizer 10 is powered by a 
primary battery circuit 36 which is comprised of several 
rechargeable batteries coupled in series, together with a 
voltage regulator, and two voltage comparators which pro- 
vide an indication of the status of the batteries in order to 45 
warn the user of the need for recharging the batteries, and so 
that the microcontroller 12 can shut down all operations 
other than maintenance of the time of day and the memory 
contents if the batteries become dangerously low. Whenever 
the batteries in the primary battery circuit 36 become low, a 5 q 
backup battery circuit 38 connects non-rechargeable backup 
batteries to power the organizer 10. If a comparator within 
the primary battery circuit 36 determines that the primary 
batteries therein are almost out of sufficient charge, the 
regulator shuts down, and only the backup batteries within 55 
the backup battery circuit 38 are used. In that instance, the 
microcontroller 12 immediately stops all operations other 
than minimal maintenance to prevent loss of the memory 
contents. A battery charging circuit 40 provides a regulated 
current to the primary batteries in the primary battery circuit 60 
36, when an external charger is plugged into a charger jack 
42. The battery charging circuit 40 automatically senses 
when a charger is plugged into the jack 42 and signals the 
microcontroller 12 accordingly. 

The DRAM 22 stores data which is generated as the user 65 
uses the organizer 10. As shown in FIG. 3, the DRAM 22 is 
divided into two basic storage areas. A first such area 44, 
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comprising the vast majority of the DRAM 22, is used for 
voice recordings. The remainder of the DRAM 22, as 
represented by a second area 46, is divided into five separate 
areas. A first one 48 of the five areas is an "overhead" storage 
area used in the operation of the personal organizer 10. The 
area 48 stores data used in maintaining the state of operation 
of the personal organizer 10. The area 48 is of fixed size, and 
the various data fields thereof are also fixed within such area. 

A second one 50 of the five areas within the storage area 
46 is used to store voice templates which are created when 
the user trains the personal organizer 10 to his or her voice. 
Because the number of words stored for recognition pur- 
poses is known, the size of the area 50 is fixed. 

A third one 52 of the five areas within the storage area 46 
contains data pertaining to reminders and memos, which are 
described hereafter. Hie area 52 is divided into 255 small 
segments, one for each memo and reminder allowed. There 
is status information indicating whether the item is a 
reminder, recurring reminder, or memo, as well as an 
indication of which recording it is associated with. The time 
of recording a memo, or the due time for a reminder, and the 
period of recurrence for a recurring reminder, are also stored 
in this area. The storage area 52 is of fixed length. 

A fourth one 54 of the five areas within the storage area 
46 contains data pertaining to the phone directory, described 
hereafter. For each entry in the phone directory, there is 
space for two voice templates for the name, together with 
space for four phone numbers which may be up to 20 digits 
in length, and an indication of which recording is associated 
with the directory entry. This storage area is also of fixed 
length. 

A fifth one 56 of the five areas in the storage area 46 * 
comprises a data table used to indicate where in the voice 
recording memory space each recording resides. This table 
is similar to file allocation tables utilized in managing disc 
drives in small computers. 

The voice recording storage area 44 is logically divided 
into fixed blocks 58 that are 512 bytes long. Only a few of 
the blocks 58 are shown in FIG. 3, for simplicity of 
illustration. Each block 58 corresponds to approximately 
one-fourth second of recording time. Each recording is 
therefore a multiple of one-fourth second in length. As a 
recording is made, the starting one of the blocks 58 thereof 
is noted in the table. The data for each reminder, memo and 
the like "points" to a unique location in the table which 
"points" to a unique one of the blocks 58 in the voice 
recording area. As new recordings are made, the first avail- 
able location in the table is assigned and the recording 
begins at the first available voice block 58. As recording 
progresses past the first one-fourth second, the next avail- 
able one of the blocks 58 is used to continue the recording. 
This block 58 may be the very next one or it may not be. At 
the end of each block is a "pointer" to the next one for this 
recording. If the block is the last one for the recording, the 
pointer indicates that this is the end of the recording. 

When a recording is erased from the storage area 44, it is 
only necessary to clear the entry thereof in the appropriate 
data area (memo/reminder or phone directory), clear the 
entry that was occupied thereby on the voice memory 
allocation table and mark the voice memory blocks 58 that 
were used, as being free. The voice data itself is not, in fact, 
erased. It is simply marked as "available", in the same 
manner as is done on discs in computers. Initially, recordings 
occupy contiguous groups of the blocks 58. However, as the 
recordings are erased and others are made, the recordings 
tend to become fragmented across non-contiguous ones of 
the blocks 58. 



11/25/2003, EAST version: 1.4.1 



5,602,963 



10 



The interna] RAM 16 within the microcontroller 12 is 
used to store information temporarily for fast access during 
each particular operation. For example, when voice recog- 
nition is to occur, the templates that are allowed for the 
particular item being accessed are pulled into the RAM 16. 5 
This is necessary because access to the RAM 16 is signifi- 
cantly faster than for the DRAM 22. As new operations 
proceed, the same memory space is reused for other pur- 
poses. 

The microcontroller 12 operates to perform voice recog- 10 
nition in the same manner as described in the previously 
referred to copending applications, Ser. Nos. 07/915,112, 
07/915,938 and 07/915,114. Such applications are incorpo- 
rated herein by reference. As described in detail in co- 
pending application Ser. No. 07/915,112 for example, the 15 
A/D converter 18 may comprise an 8-bit converter which 
samples incoming data at a preassigned frequency such as 
9.6 KHz. In that event, the A/D converter 18 outputs a digital 
signal representing the input analog voice signal. A micro- 
processor (MP) 62 within the microcontroller 12 processes 20 
the digital voice signal together with a voice recognition 
software routine forming part of a control program stored in 
the ROM 14. The digital voice signal is converted into an 
incoming voice template that is compared against previously 
stored voice templates of the user's voice, stored in the 25 
external DRAM 22. The program decodes the voice tem- 
plates. Together with the external DRAM 22, the RAM 16 
comprises a reference memory for temporary storage of 
data. 

Thus, the analog voice signal is applied to the A/D 30 
converter 18 for conversion into an incoming digital voice 
signal. The reference memory, comprised of the DRAM 22 
in conjunction with the RAM 16, stores a plurality of 
reference digital voice templates. The ROM 14 stores the 
control program. The microprocessor 62 which is coupled to 
the A/D converter 18, the ROM 14 and the RAM 16 
generates an incoming digital voice template from the 
incoming digital voice signal at the output of the A/D 
converter 18. The microprocessor 62 then executes the 
control program to determine whether the incoming digital 
voice template is substantially equivalent to one of the 
reference digital voice templates, stored in the reference 
memory comprised of the RAM 16 and the external DRAM 
22. The microprocessor 62 determines what action to take 
corresponding to a reference digital voice template, if the 45 
incoming digital voice template is found to have substantial 
similarity to the reference digital voice template. 

Voice control is made possible by first voice training the 
collection of reference digital voice templates in accordance 5Q 
with the user's voice. Such templates are collected in the 
same manner as described in co-pending application Ser. 
No. 07/915,112. When voice training is complete, the per- 
sonal organizer 10 is ready for use. 
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The basic program for the organizer 10 begins with an idle 
mode/select operation mode software routine, shown in the 
flow diagram of FIG. 4. When in such idle mode, as 60 
represented by a block 64, the clock and calendar are 
displayed by the LCD 20 as shown in FIG. 2. As shown in 
FIG. 4, various different operations may take place from the 
idle mode 64, including memo record 66, calendar (voice 
access) 68, phone directory 70, memo review (play last 65 
memo) 72, voice training 74, set clock 76, waiting message 
78, and security 80. 



As shown in FIG. 4, by pressing the silent button from the 
idle mode 64 memo review 72 may be selected, in which 
event the last memo is selected. Selection may also be made 
of reminder review 84, in which event the first reminder is 
selected. Further selections include recurring reminder 
review 86, in which event the first recurring reminder is 
selected, calendar 88, with today being selected, and calcu- 
lator 90. The memo review 72, the reminder review 84 and 
the recurring reminder review 86 modes are described 
hereafter in connection with the flow diagram of FIG. 11. 

As previously noted, the idle mode 64 shown in the flow 
diagram of FIG. 4 includes display of the current time of 
day, the date, and the day of the week, using the LCD 20. All 
operations other than those necessary to maintain the time of 
day, the memory contents, and to monitor the key states, are 
disabled, to minimize drain on the batteries. The idle mode 
64 is exited if a key of the keypad 34 is pressed or if a 
reminder time comes due. 

Topically, the first thing a user does with the organizer 10 
is to set the clock to the current time and date (set clock 76) 
and then train the organizer to his or her voice (voice training 
74). Once these items are completed, all other features will 
be fully operational. Until the voice training is completed, 
none of the voice recognition features will operate. Instead, 
they will result in a message 'TRAIN" being displayed, to 
prompt the user to complete voice training. 

SET CLOCK (FIG, 5) 

The set clock function 76 of FIG. 4 is shown in detail in 
the flow diagram of FIG. 5. The user begins by holding the 
"edit" button for two seconds while in the idle mode 64. A 
simple press of the "edit" button results in a single beep 
being generated by the personal organizer 10, and this is 
used for editing. To prevent the clock setting procedure from 
inadvertently being initiated, the user must hold the "edit" 
button for a full 2 seconds until a subsequent double beep is 
heard. At this point, the current time and date are displayed 
with the hour field blinking. The user then presses the "next" 
button to advance the hour by one or the "prev" button to 
decrease the hour by one. When the correct hour is displayed 
(including AM or PM), the user presses the "edit" button to 
again move to the day field. The day field is edited in the 
same manner. The user again presses the "edit" button to 
move to the next fields, namely month, day and year, using 
the "next" and the "pref ' buttons to change the entries, until 
the entire date and time are entered. At any point, the user 
can press the "save" button to accept the date and time as 
entered, or the "stop" button to revert to the time and date 
that were set before the user began the operation. 

Thus, as shown in FIG. 5, pressing of the "edit" button for 
at least two seconds, while in the idle mode 64, initiates a 
series of operations including set hours 92, set minutes 94, 
set month 96, set day 98, set year 100, and display all 102. 
During each of the operations 92, 94, 96, 98, 100 and 102, 
pressing of the "next" button advances the display by one, 
while pressing of the "prev" button decreases the display by 
one. Each pressing of the "edit" button moves to tie next 
field or operation along the chain of operations 92, 94, 96, 
98, 100 and 102 for editing thereof. Pressing of the "save" 
button accepts the date and time as entered, as represented 
by a save time/date block 101. Pressing of the "stop" button 
reverts to the time and date that were set before the user 
began the operation, as represented by a no change block 
103. 

VOICE TRAINING (FIG. 6) 

The voice training function 74 of FIG. 4 is shown in detail 
by the flow diagram of FIG. 6. Voice training is initiated by 
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pressing the ''train" button while in the idle mode 64. If the 
user has not yet performed initial complete training, then 
complete training is begun. Otherwise, a "train one-word" 
process is begun. 

Upon commencement of initial complete training, as 5 
determined in a step 104 in FIG. 6, the first word of the 
training list is displayed. The user holds the "time" button 
and speaks the word which is displayed. The template of the 
spoken word is stored, and the next word is displayed, in a 
step 106. The process continues until the last word is stored 10 
in this manner, as determined in a step 108. 

When the template for the last word in the list is collected, 
a second template for each word is collected in the same 
order, as determined in a step 110. The collected template is 
compared to the previously collected template, in a step 112. 15 
If the template compares best with the previous template for 
the same word, as determined in a step 114, then it is saved 
and the organizer 10 progresses to the next template. If it 
does not match the previous template for this word, then the 
user is asked to repeat the word for collecting a third 20 
template, in a step 116. The process repeats until there are 
two templates for each word that match. When this has been 
accomplished for all words in the list, the organizer 10 
returns to the idle mode 64, in a step 118. 

25 

If the train mode is entered when there are existing 
templates, then the user may train a word of his or her 
choice, as represented by a step 120. This is useful to correct 
problem words that are not being recognized properly. The 
user presses either the "next" or the "prev" button to scroll 3Q 
through the list of words. When the word the user wants is 
displayed, the user proceeds to hold the "time" button and 
collect two templates, as represented in steps 122, 124, 126, 
128 and 130. 

If the user wants to completely retrain ail templates, then 35 
after entering the train mode, the user holds the "erase" 
button until a double beep is heard, as represented by a step 
132. All templates are erased and the complete training 
process is initiated. 

MEMO RECORD (FIG. 7) 40 

Memo recording can be accessed from several states in 
the overall program flow, as shown in the memo record flow 
diagram of FIG. 7. At these points in the program flow, the 
user holds the "record" button and speaks into the organizer 45 
10. The microcontroller 12 activates the analog circuitry 
including the voice compression and decompression circuit 
28, and begins to store the incoming recording data in the 
DRAM 22. This is represented in a step 134 in FIG. 7, in 
association with the idle mode 64, the memo review 72, the 50 
reminder review 84, the recurring reminder review 86, and 
the calendar 88. If the "record" button is released before 0.75 
seconds has elapsed (i.e., before three of the blocks 58 
shown in FIG. 3 are filled), then the recording is "erased" 
automatically and the idle mode is returned to, as repre- 55 
sented by a step 136 in FIG. 7. Conversely, if the recording 
is longer than 0.75 seconds, then the recording is added to 
the data memory and the memo review mode 72 is returned 
to with the new memo selected, as represented by a step 138. 
The 0.75 second minimum time is implemented to prevent 60 
unintentional "silent" memos from being recorded due to 
accidental pressing of the "record" button. 

REMINDER SETTING (FIG. 8) 

Reminders are built on memos. A due time (alarm time) 65 
is added to a memo by the user. When the due time arrives, 
the organizer 10 alerts the user with a short series of beeps, 
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as described hereafter. When a memo is recorded, it can 
immediately be turned into a reminder, A memo may also be 
turned into a reminder at any later time, simply by selecting 
the memo, 

FIG. 8 is a flow diagram of the reminder setting, which 
shows the process of creating a reminder. The user begins by 
selecting the memo to be made into a reminder (or an 
existing reminder for which a new due time is desired) and 
then holding the "time" button, as represented by a step 140. 
The organizer 10 then prompts the user for the information 
needed, so that the user does not have to remember the exact 
procedure required. First of all, the current time and day are 
displayed with the hours field blinking to indicate that the 
user should speak the hour. See the display shown in FIG. 
19A. If the user speaks the word "timer", then the process 
changes to a timer input, as described below. For a reminder, 
the user recites the number for the hour desired (1-12), in a 
step 142. The hour recognized is displayed and the minutes 
field along with the AM/PM is selected and proceeds to 
blink, as shown in FIG. 19B. This is represented by a step 
144 in FIG. 8. The user may stale either a one-fourth hour 
(15, 30 or 45) or "AM" or "PM" if the desired time is on the 
correct hour. If a one- fourth hour is stated, then the AM/PM 
which is chosen blinks, and the user must state either "AM" 
or "PM". This is represented in a step 145. 

At this point, the time setting is complete. During the 
preceding entry process, and for the remaining data entry, 
there is a 0.5 second time-out that will occur if the user 
releases the "time" button. If the user releases the "time" 
button for longer than 0.5 seconds, then the reminder time is 
either saved as entered or, if the entered data is incomplete, 
then no change is made and the item reverts back to its 
former state, as represented by a step 146. The minimum 
entry required is as previously described, up to and including 
AM/PM. 

If the user stops after AM/PM entry, then the reminder is 
saved and said to alert the user at the next occurrence of the 
time entered (either later today or the next day). 

The user may continue to hold the "time" button and enter 
a day or date, as represented in a step 148. At this time, the 
day field blinks and "DAY-DATE" is displayed to indicate to 
the user that either a day (Sunday through Saturday) or an 
entire date (1 1/25 for example) may be entered, as shown in 
the display of FIG. 19C. The user may also say the word 
"recurring" at this point, to make a daily recurring reminder, 
as discussed below. If a day of the week is recognized, then 
date entry is completed and the reminder will alert the user 
at the next occurrence of the time and day entered. 

If a number is entered, it is placed in the month field and 
the day field continues to blink, as shown in FIG. 19D. The 
user proceeds to enter the day of the month, one digit at a 
time. If the day is the 1st through the 9th, then the user stops 
after speaking one digit. If the day is 10 or greater, the user 
must speak each digit individually. Unless the date requested 
is invalid, the reminder is saved as entered, as shown in the 
display of FIG. 19E. 

At the end of any valid time/date entry, the user may say 
the word "recurring". The rerninder is thereby made a 
recurring type reminder, and a period of recurrence is 
assigned based on the amount of information entered. If the 
user stops after entering AM or PM, the word "daily" is 
displayed and blinks. If a day of the week was entered, the 
word "weekly" is displayed and blinks. If the entire date was 
entered, then the word "monthly" is displayed and blinks. 
The user can press the "next" or "prev" buttons to scroll the 
allowable periods (daily, weekly, monthly or yearly). When 



11/25/2003, EAST 



version: 1.4.1 



5,602,963 



13 



14 



10 



the desired period is displayed, the user can press the "save" 
button to record this entry, or wait for a 10-second time-out 
to automatically save it. 

The day portion of a date entry is represented in steps 150 
and 156. The "recurring" functions are represented in steps 
152, 154, 156 and 158. Each of these steps involves spoken 
words (shown in quotes) or a time-out, as described above, 

MANUAL ENTRY (FIG. 9) 

The user may use a manual method of entering a reminder 
time/date rather than voice input. Manual entry, as shown in 
the flow diagram of FIG. 9, begins with the pressing of the 
"edit" button. The process is similar to that for setting the 
clock time previously described. Namely, the user scrolls 15 
through the choices for each field, using the "next" and the 
"prev" buttons to increase or decrease each field by one. The 
user moves on to the next field by pressing the "select 1 ' 
button. 

Thus, as shown in FIG. 9, the pressing of the "edit" button 20 
after recording a memo or with a memo or reminder 
selected, begins the manual entry process as represented by 
a step 170. The hour field is entered for editing, as repre- 
sented by a step 172. The editing process begins at the 
current time, and the "next" and "prev" buttons are used to 25 
increase or decrease the hour by one, respectively. Pressing 
of the "select" button moves to the edit minutes field as 
represented by a step 174. Following editing of the minutes, 
again using the "next" or "prev" buttons, the user uses the 
"select" button to advance to the edit months field, as 30 
represented by a step 176, and then to an edit day field as 
represented by a step 178. Pressing of the "select" button 
one more time returns to the step 172. 

As with voice entry, only complete and valid entries are 
saved. Invalid or incomplete entries revert to their former 35 
state. Once a valid entry is inputted, the user can hold the 
"edit" button for a double beep to make this a recurring 
reminder. The user can then choose the period of recurrence 
in the same manner as is done for voice input. As represented 
by a step 180 in FIG. 9, pressing of the "save" button in any 40 
of the steps 172, 174, 176 or 178 saves the reminder by 
going to reminder review with the reminder selected. Press- 
ing of the "stop" button within any of the steps 172, 174, 176 
and 178 indicates no change, in which event the system goes 
to memo or reminder review with the item selected, as 45 
represented by a step 182. 

TIMERS (FIG. 10) 

Timers are a variation of reminders. The user can set a 5Q 
timer to go off in a period of time measured from the 
moment the timer is set, up to 23 hours and 59 minutes. 

A timer setting is begun with the user saying the word 
"timer" as the first word in setting a reminder. The process 
is shown in the flow diagram of FIG. 10. From the reminder 55 
setting flow diagram of FIG. 8, which is represented by a 
block 190, saying the word "timer" leads to a step 192 in 
which the user states a one- or two-digit number, and follows 
it by stating "hour*', "hours" or "minutes" as represented in 
steps 194, 195, 197, 198, 199 and 201. If the user sets a 60 
number of hours first, the user can follow with a second one- 
or two-digit number, followed by stating "minutes". 

As with reminders, when the "time" button is released, 
there is a 0.5 second time-out after which the timer is saved. 
As shown in FIG. 10, a time-out with an incomplete entry 65 
results in a no change step 196, in which event the system 
goes to memo or reminder review with the item selected. On 



the other hand, a time-out with a complete entry at steps 197 
or 205 results in a save reminder step 203. 

After the user states "hour", "hours" or "minutes", the 
user may proceed to state "recurring". A recurring timer 
alerts the user with a period of recurrence equal to the timer 
setting. For example, if the timer is set for 8 hours, the user 
is alerted every 8 hours from the time that the umer is set. 

Manual entry for the timer is initiated for manual entry of 
a reminder. After pressing the "edit" button to start manual 
entry, the user presses "edit" again to change to the timer 
format. As with reminders, the user presses "next" or "prev" 
to set the desired number for each field, and presses "select" 
to move from the field. In this case, there are only two fields, 
hours and minutes. A timer can be made recurring manually 
by holding the "edit" button for a double beep. 

MESSAGE REVIEW (FIG. 11) 

As described in connection with the idle mode/select flow 
diagram of FIG. 4, the user can review his or her messages 
by type, in terms of memos, reminders and recurring 
reminders. As represented by a block 200, in FIG. 11 which 
is a message review flow diagram, the user may press the 
"select" button to reach the desired category. From the idle 
mode (clock display), the user presses once to get to memos, 
twice to get to reminders and three times to get to recurring 
reminders. There is also a shortcut for memos from the idle 
mode, achieved by the user pressing "play" with the memo 
category being selected and the last memo recorded being 
automatically played back. 

When the memo category is selected, the last memo to be 
recorded is defaulted to. Because memos are numbered in 
the order that they are recorded, such memo will be the Nth 
memo, where N is the number of memos presently saved. 
When the reminder or recurring reminder categories are 
selected, the next reminder or recurring reminder to come 
due (i.e., 1) is defaulted to. The message review flow 
diagram of FIG. 11 demonstrates this process. 

Once the category is selected, the user has many options. 
Pressing "stop" exits back to the idle mode. Pressing 
"select" proceeds to the next category. Pressing the "record" 
button begins the recording of a new memo. Pressing the 
"time" button sets a reminder time for the message selected. 
Pressing the "edit" button begins manual entry or editing of 
a reminder time. Holding the "erase" button for a double 
beep erases the message. 

Only the "play", "prev" and "next" buttons are used for 
actual review of the messages. The user presses "play" to 
play back the selected message. Pressing "next" advances to 
the next message, automatically playing it Pressing the 
"prev" button moves to the previous message, automatically 
playing it. The user does not have to wait until the playback 
of a message is complete before moving to the next or the 
previous message. Pressing the "next" or "prev" button 
during playback aborts the playback and moves to the next 
or previous message, automatically playing it back. The 
selection of messages "wraps around". In other words, if the 
last memo is selected, pressing "next" moves to the first 
memo. Likewise, if the first one is selected, pressing "prev" 
moves to the last one. 

While a message is being played back, the user can press 
the "stop" button to pause the playback. The user presses 
"play" to resume playback where it was paused. Pressing 
"stop" while playback is paused will cause the tl pause" 
display to be removed and the current memo to still be 
selected. Pressing "play" at this point starts playback from 
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the beginning of the message. Any time a message is 
displayed but is not being played back and is not paused, the 
user can press "stop" to return to the idle mode. 

At any point in the review process where a message is not 
being played back, there is a 30-second time-out that causes 
the same action that would occur if the stop button were 
pressed. The flow diagram of FIG. 11 applies to memo 
review, reminder review, and recurring reminder review. 
When entered using the "select" button, the last recorded 
memo or next reminder or recurring reminder to come due 
is defaulted to. When entered by creation/editing of a 
message, that message is defaulted to. As noted, message 
. numbers are assigned and messages are reviewed chrono- 
logically. Memos are done by the time recorded, while 
reminders and recurring reminders are done by the time that 
they are to come due. Timers and recurring timers show 
count-down to the time they will come due. The text 
displayed depends on the message type. Memos display 
"RECORDED", reminders display "PLAYBACK", timers 
display "PLAY IN", recurring reminders display "DAILY", 
"WEEKLY", "MONTHLY", or "YEARLY". 

From the block 200 of FIG. 11, pressing the "stop" button 
reaches the idle mode 64. Pressing the "select" button results 
in selection of the next category, represented by a step 202. 
Pressing the "record" button results in memo record 66. 
Pressing the "time" button sets a new time, as represented by 
a step 204. Pressing the "edit" button sets the new time 
manually, as represented by a step 206. 

From the block 200, pressing of the "play" button results 
in playing of the message, as represented by a step 208. 
Pressing of the "next" button proceeds to the play step 208 
through a step 210 in which the next message is selected. 
Pressing the previous or "prev" button proceeds to the 
playing step 208 through a step 212 in which the previous 
message is selected. Pressing of the "stop" button results in 
the pause of playback, as represented by a step 214. Pressing 
the "play" button continues playback, as represented by a 
step 216. At this point, however, pressing of "stop" returns 
to the block 200. Pressing of "erase" at any of the various 
steps results in erasure of the message and selection of the 
next message, as represented in a step 218. 

WATTING MESSAGES (FIG. 12) 

Whenever a reminder or a recurring reminder (including 
timers) comes due, the organizer 10 generates a short 
sequence of beeps to alert the user. The user is locked out of 
all other operations until he or she either listens to all waiting 
messages or defers them. If the user does not respond, the 
beep sequence repeats as follows: 

1. Every 5 minutes until 15 minutes past due, 

2. Every 15 minutes until 2 hours past due. 

3. Every hour until 12 hours past due. 

4. After 12 hours, there is no more beeping. 

This is shown in FIG. 12 which is the waiting message flow 
diagram. 

As shown in FIG. 12, from the idle mode 64 a step 230 
occurs when a reminder comes due. There are seven sets of 
two beeps, 0.5 seconds apart. After the beeps have occurred, 
the word "MESSAGE" is displayed and blinks, as repre- 
sented by a step 232. The "message" display is shown in 
FIG. 19F. Every time a minute passes, the organizer 10 
automatically checks to see if other messages have come 
due, as represented in a step 234. If one or more messages 
have come due, then the organizer 10 beeps again, updates 
the number of messages on the display and returns to the 



"message" display. When this display occurs, the user has 
two options. 

Typically, the user will press the "play" button to begin 
review of the waiting message or messages, as represented 
in a step 236. The display is shown in FIG. 19G. As each 
waiting message is played, the user can use the "next" and 
"prev" buttons to move from message to message. The only 
difference between review of waiting messages and review 
of messages that have not come due is that, for waiting 
10 messages, there is no automatic playing of the message 
when the "next" or ''prev" buttons are pressed. Selection of 
the next waiting message by pressing of the "next" button is 
represented in a step 238. Selection of the previous waiting 
message by pressing the "prev" button is represented by a 
15 step 240. 

The user can also set a new playback time for a reminder 
after reviewing it. This is done by either holding the "time" 
button and entering a due time by voice, as represented by 
a step 242, or by pressing the "edit" button and entering the 

20 new time manually, as represented by a step 244. 

When playback of a message is completed, there is a 
30-second time-out after which the review process is exited. 
Pressing the "stop" button has the same effect. Upon exiting 
the review mode, if there are still messages that have come 

25 due that have not been reviewed, the "MESSAGE** display 
is returned If all waiting messages have been reviewed, then 
the idle mode 64 is returned to. 

A second option which the user has is to defer the waiting 
messages by pressing the "stop" button. This effectively sets 

30 all of the waiting reminders to occur again in 15 minutes, 
thereby freeing the features of the organizer 10 for user 
access. Deferring is represented by a step 248. If the user 
wishes to defer for a different period of time, he or she holds 
the "stop" until a double beep is heard, as represented in a 

35 step 246, and "SET DEFER" is displayed. The user then 
keys in a new defer period in the same way that a timer is 
set manually. 

The user may also change the default deferral time of 15 
minutes by holding the "time" button for a double beep at 

40 the idle clock display and setting a new default defer time in 
the same manner. 

As noted, from the play message step 236, pressing of the 
"erase" button erases the reminder, in a step 250. Pressing 
the "save" button saves the recording as a memo, as repre- 

45 sented in a step 252. Pressing "stop" deletes reviewed 
reminders that have not been saved, as represented in a step 
254. Inactivity for 30 seconds has the same effect, in this 
instance, as pressing the "stop" button. In the step 254, 
recurring reminders are not automatically deleted. Whether 

50 or not there are unreviewed waiting reminders is represented 
by a step 256. 

CALENDAR (FIG. 13) 

55 The calendar mode, as shown in the flow diagram of FIG. 
13, allows the user to review all of the reminders and 
recurring reminders for a single day combined. As shown in 
the idle mode flow diagram of FIG. 4, there are two ways to 
access the calendar. 

60 The calendar may be accessed by pressing the "select" 
button as represented in a step 270. The calendar follows the 
review of recurring reminders in the select process. When 
the calendar is selected, the display, as represented by a 
block 272 shows the present date and the day and the 

65 number of reminders and recurring reminders for that day. 
The user can either press the "next" button to move forward 
to another day, as represented in a step 274, or hold the 
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"time" button and speak a day or date, as represented in a 
step 276 to go to another day. When at another day, the user 
can use the "next" or "prev" buttons to move around from 
day to day, or the user can hold the "time" button to do so 
by voice. Moving to the previous day by pressing the "prev" 
button is represented in a step 278. Holding of the "time" 
button provides a shortcut from the idle mode. The user 
simply holds the "time" button and states the day or date 
desired. 

The only dates shown for the calendar are from the 
present day to a year in the future (less one day). Scrolling 
with the "next" and "prev" buttons stops at these extremes. 
By voice, the days of the week cause the calendar to "jump" 
to today or one of the following six days. A date is entered 
in the same manner as is done for reminders. The numbers 
1-12 are used for the month, followed by a one- or two-digit 
day. This is represented by steps 280, 282, 284, 286 and 288. 

When the day desired by the user has been selected, the 
user may press "play" to effectively enter the day, beginning 
with the playing back of the first reminder or recurring 
reminder to come due on that day, as represented in a step 
290, and with the due time and other data shown on the 
display as is done in the review of messages previously 
described. In this case, the messages begin with the first one 
through the number due for that day, according to the time 
that they are due. The user moves from message to message, 
using the "prev" and "next" buttons. The newly-selected 
message is automatically played back in the usual fashion. 

When the user is finished reviewing the messages for a 
particular day, pressing of the "stop" button returns to the 
display of the date and day, thereby allowing the user to 
select another day, as before. 

PHONE GROUP SELECT (FIG. 14) 

The phone directory is accessed for both input and output 
using the "phone" button, primarily. To optimize voice 
recognition, several alphabetical groups are assigned. Each 
name is input into one of the alphabetical groups and is 
retrieved from the same group. The groups are chosen for 
equal average utilization. For example, five groups of twenty 
names are alphabetically divided as follows: A-D, E-I, J-M, 
N-R and S-Z. 

This is shown in the phone group select flow diagram of 
FIG. 14, which illustrates the manner in which the individual 
alphabetical groups are accessed. From the idle mode 64, the 
user presses the "phone" button multiple times to cycle 
through five different alphabetical groups of names 292, 
294, 296, 298 and 300. For example, one press of the 
"phone" button accesses the A-D group 292. A further press 
of the button accesses the E-I group 294. Any time the 
"stop" button is pressed or a time-out occurs, the phone 
directory is exited. 

PHONE NUMBER RETRIEVAL (FIG. 15) 

FIG. 15 is a flow diagram for the phone number retrieval 
process, which accesses phone numbers already entered into 
the directory. From the" idle mode 64, the alphabetical groups 
are displayed in a start step 310, and pressing of the "phone" 
button advances through the various alphabetical groups, as 
represented by a step 312 in FIG. 15. This takes place as 
previously described in connection with FIG. 14. The most 
common use of the phone directory involves holding the 
"phone" button down once the desired group is displayed, 
and speaking the name desired. This is represented by a step 
314. When the name is recognized, the first number asso- 



10 



20 



25 



30 



35 



40 



45 



50 



55 



60 



65 



ciated with the name is displayed. If the user wants another 
number associated with this name, the user presses the 
"next" key to get the next number, in steps 320 and 328. If 
there is no next number for the name, the next name in the 
directory is accessed, in a step 330. Every time a number is 
accessed, the recording associated with the directory entry is 
played back to confirm the name selected. The number is 
displayed at the same time, as represented by a step 318. 
Pressing the "play" button from the start step 310 selects the 
first number for the first entry, as represented by a step 318, 
for playback recording in the step 316. 

From the step 316, the user can navigate manually 
through the directory using the "next" and "prev" buttons, as 
represented by steps 320 and 322. From the alphabetical 
group display, the user can access the directory without ever 
speaking a name by pressing the "play" button. This 
accesses the first number of the first name in the directory, 
as represented in the step 318. Any time a number is 
displayed, pressing the "play" button results in replay of the 
recording for that entry. 

At the number display, represented by the step 316, the 
user may press the "stop" button to return to the alphabetical 
group display, via the start step 310, or press and release the 
"phone" button to go to the next group, as represented by the 
step 312. 

■The user may also perform editing features at the number 
display of the step 316. The user can retrain the templates for 
better recognition by pressing the "train" button to retrain 
the name in a step 324. This will initiate the same training 
sequence that was performed when the number was entered. 
The new templates replace the old templates, once the 
retraining is completed. If the user presses the "edit" button, 
as represented by a step 326, the first. digit of the displayed 
phone number blinks and the user can change it using the 
"next" and "prev" buttons, as has been described for other 
manual entry. The user advances from digit to digit while 
editing, by pressing the "select" button. The recording for 
the selected entry can be replaced by holding the "record" 
button for a double beep and speaking the new utterance to 
be recorded, in a step 327. 

Another number may be added to an entry by holding the 
"edit" button for a double beep as represented in a step 336 
with a number associated with the desired entry being 
displayed If the "edit" button is held for a double beep while 
an alphabetical group is displayed, then a new entry is made. 
If a new name is being added to the directory, the "edit" 
button is held at the alphabetical group display at the start 
step 310. If the group is full, an error message is displayed 
briefly and the group display is returned. If the group is not 
full, the screen prompts the user for the name. The user holds 
the "phone" button and speaks the name, repeating as 
prompted. Once the name is trained, the display prompts for 
the number. (If the "edit" button was held while a phone 
number is displayed and there arc not four numbers already 
associated with this entry, then a new number is added 
starting at this point in the flow.) The user continues to hold 
the "phone" button and enters the phone number, one digit 
at a time. 

When the user is finished entering the phone number, he 
or she presses the "save" button to indicate completion. This 
is necessary to allow for a variable length phone number 
(i.e., with area codes or without, and international numbers). 
The user then selects which of the four types of phone 
numbers is present; business, home, fax or other. Only those 
not already filled for this entry are presented to the user, who 
chooses using the "next" and "prev" buttons, and terminates 
by pressing the "save" button again. 



11/25/2003, EAST version: 1.4.1 



5,602,963 



19 



20 



The display then prompts the user to record a message. 
Typically, the message will simply be the name of the 
person, although a more detailed message may be recorded 
with other information. This recording is played back any 
time any of the numbers associated with this entry arc 
played. 

ADD PHONE NUMBER (FIG. 16) 

The process of adding a phone number to the directory, as 
just described, is shown in the flow diagram of FIG. 16. The 
alphabetical group display of the start step 310 of FIG. 15 is 
represented by a step 350 in FIG. 16. Holding the "edit" 
button for a double beep reaches a step 352 to determine if 
there are already twenty entries in the directory. If so, then 
the step 350 is returned to via a step 354. If not, then first and 
second name templates are collected in steps 356 and 358, 
and the phone number is input in a step 360. The template 
collection process is the same as that used for training, as 
previously described, except that the "phone" button is held 
for input and repeat of the second collection as necessary. A 
time-out during the steps 356, 358 and 360 or pushing of the 
"stop" button during the step 360 causes return to the 
alphabetical group display step 350. 

From the step 360, pushing the "save" button advances to 
a step 362 to select the type of phone number such as 
business, home, fax or other. From the step 362, a time-out 
results in an accept default step 364. After release of the 
"phone" button, the time-out period for template collection 
is 0.5 seconds. All other time-outs result after 30 seconds of 30 
inactivity. 

From the step 362 or the step 364, recording is performed 
in a step 366. Recordings are done by holding the "phone" 
button or the "record" button. Again, pressing the "stop" 
button at either of the steps 362 and 366 returns to the 
alphabetical group display step 350. Once recording in step 
366 is completed, the phone number is displayed, as repre- 
sented by a step 368. 

If the process is started by holding the "edit" button from 
a phone number display rather than an alphabetical group 40 
display, then the template collection is bypassed and a new 
number is added to the entry already selected. A step 370 
determines whether or not there are four numbers already 
associated with an entry. If there are not four numbers 
already associated with the entry, then the new number is 
added starting at this point in the flow, via the step 360. 
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SECURITY (FIG. 17) 

The security mode allows the user to lock-out all opera- 50 
tions of the organizer 10 until a secret 4-digit code is entered. 
The security mode is accessed from the clock display of the 
idle mode 64 by holding the "select" button for a double 
beep. The first time that this is done by the user, the only 
activity permitted is establishment of a security code. Any 55 
other activity returns to a "SECURITY OFF' display in a 
step 380. To establish a code, the user holds the "edit" button 
for a double beep and then enters a 4-digit code in a step 404. 
Entry by voice is done by holding the "time" button and 
speaking four digits, each in the 0-9 range. Entry is done ^ 
manually by pressing the "edit" button and then using the 
"next", "prev" and "select" buttons as for other manual 
functions. Once entry is complete, the "SECURITY OFF* 
display is returned, at the step 380. 

With the code established, the user can move from the 65 
"SECURITY OFF* display, in step 380, to a code entry 
display for activating security. This is done by pressing the 



"select" button, in a step 386. With a security code having 
been established, the code entry is displayed, to turn security 
on, in a step 388. The user then holds the "time" button and 
speaks the four digits of the code in a step 390. Alternatively, 
the user can press the "edit" button and proceed with manual 
entry, in a step 392, as in the case of the step 384. 

If the code entry is correct, the display momentarily 
shows the "SECUR ON" and then returns to the idle mode, 
but with security activated, in steps 394 and 396. If the entry 
is incorrect, the display shows "WRONG" and the unit is 
locked-up for 30 seconds, in a step 398. 

Once security is activated, any keypress will go directly 
to the code entry display for entry of the code to deactivate 
security, as represented by a step 400. The user enters the 
code as previously described. If the entry is correct, the 
display shows "SECURITY OFF 1 and then returns to the 
idle mode, via a step 402. 

When security is off, the user can change the security code 
by holding the "edit" for a double beep from the "SECU- 
RITY OFF* display 380. The user is asked to enter the old 
code. If the entry is correct, then the user is asked to enter 
a new code, via a step 404. New code entry can be performed 
by voice or Manually, as with other code entries. Any time 
the user enters an incorrect code, the system locks up for 30 
seconds while displaying "WRONG" and then returns to the 
idle mode 64, as represented in steps 408 and 398. 

"NO" LOGIC (FIG. 18) 

Any time a voice recognition feature is used, the user has 
a means to correct for a misrecognized word. When the 
display shows a number or word that was not the word the 
user spoke, the user may subsequently say "no" while still 
holding the "time** (or "phone") button. This is shown in the 
"no" logic flow diagram of FIG. 18. At this point, the 
organizer 10 displays the word or number that scored second 
highest during the recognition process. If this is still not 
correct, the user may say "no" again. This time, the orga- 
nizer 10 back-spaces over the entry and encourages the user 
to repeat the word. If the word is still misrecognized, the 
user may continue saying "no" and this process will iterate, 
with the second best guess being followed by repeating the 
word. Once the user is satisfied that the number or word 
displayed is correct, the user continues on with the next 
word or number desired. 

This is shown in the flow diagram of FIG. 18 with a 
speech input at a step 420. This results in displaying of the 
word or number with the highest score, in a step 422. This 
is followed by speech input for the next item or the end of 
speech input, in a next step 424. If the user enters "no" then 
the word or number with the second highest score is dis- 
played, in a step 426. Otherwise, the word or number for the 
next item is displayed, or speech input is exited in a step 428. 

Following display of the word or number with the second 
highest score, in the step 426, speech input for the next item 
is entered or is ended, in a step 430. At this point, if the user 
speaks "no", then "REPEAT" is displayed, in a step 432, and 
the system returns to the step 420. Otherwise, the step 428 
is advanced to. 

The utilization of voice recording and voice recognition 
technology, in voice-activated personal organizers 10 in 
accordance with the invention, allows the size of the orga- 
nizer 10 to be much smaller than manually-operated devices. 
The particular implementation of the organizer 10 shown in 
FIG. 2 is somewhat oval in shape so as to comfortably fit 
into the palm of the user's hand in such a way that the 
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organizer 10 can be operated with one hand. The "record" 
and "time" buttons are on the corners immediately acces- 
sible by the user's thumb and forefinger. The "phone" and 
"select" buttons are on the top, between such buttons. These 
four buttons are the most-used buttons. 5 

The "play", "next", "play" and "stop" buttons are on the 
face of the organizer 10, below the LCD display 20, and in 
a diamond-shaped arrangement. These buttons carry the 
universal symbols used in tape players and similar devices. 
The four least used buttons are the "train", "save", "erase" 10 
and "edit" buttons These buttons appear below the "play", 
"next", "play", and "stop" buttons. 

The microphone 24 and the speaker 32 are located behind 
the buttons on the face of the organizer 10, and face in the 
same direction as in the LCD display 20. This ensures proper 15 
alignment when the user places the unit for easy viewing of 
the display 20. 

It will be appreciated by those skilled in the art that 
variations and alternatives are possible in accordance with 
the invention. For example, the organizer 10 can be embod- 20 
ied in a form more closely related to that of a standard 
organizer with a small keypad for entering some text that is 
beyond current voice recognition technology, such as 
addresses. In such instances, the use of voice recognition 
technology for most input would make it feasible to use a 25 
smaller keyboard (and thus a smaller physical unit), without 
making the organizer difficult for the user. Such an organizer 
can have a larger display, thereby allowing for more com- 
plex messages of the type used in larger organizers. Such 
complex images can include, for example, a full week 30 
calendar showing appointments "blocked out" or names 
with addresses and phone numbers. This variation provides 
the user with the benefits of a more natural interface using 
voice for recordings and recognition, together with the 
ability to input more complex information using a keyboard 35 
when necessary. 

Still other variations of the organizer 10 in accordance 
with the invention can be implemented in a physical form 
similar to that shown in FIG. 2, but with some of the 
software features being changed. For example, an expense 40 
logging and reporting feature can be added with no change 
to the physical or electrical implementation. In such a case, 
the user can access the expense feature in the same way as 
the calendar described herein. The user either presses the 
"select" button several times to access expenses manually, or 45 
presses the "time" button (probably renamed "voice" for this 
application) and says the word "expenses." After entering 
the expenses category, the user can speak the name of a 
category such as "air fare", "car rental", "entertainment", 
"hotel", "meals", "mileage", "phone", or the like. To add an 50 
expense item, the user continues to hold the "voice" button 
and speaks a dollar value, one digit at a time, such as "two 
. . . one . . . point . . . seven . . . five" for $21.75. Once the 
entry is completed, the user can record a comment to be 
attached to such entry, in the same way that a recording is 55 
attached to a phone directory entry. When the item is saved, 
it is tagged with the current time and date. For retrieval of 
information, the user holds the "voice" button and says the 
word "expenses" followed by the name of a category. 
Instead of continuing to speak, the user releases the "voice" 60 
button. The organizer then shows the total number of items 
recorded for this category. At this point, the organizer 
functions in much the same way as in the case of the 
calendar feature described above. The user presses the 
"play" button to start review of the entries, one by one, for 65 
this category in chronological order, moving from entry to 
entry using the "next" and "prev" buttons. At the end of all 



entries is the total for such entries, so that the user does not 
have to add up all the entries by himself or herself. 

When finished with the entries in a category, the user 
presses the "stop" button to return to the category display, 
and then uses the "next" and "prev" buttons to move to 
another category or "stop" (again), to exit. In this way, the 
user can easily review all entries in each category for filling 
out an expense report, and the like. A special "category" is 
available by voice or manually. For example, for a total, the 
user either speaks the word "total" or manually advances 
through all the standard categories to arrive at the total 
which will display the grand total of all entries in the 
expense categories. 

Another special "category" is "date". If the user says the 
word "date" instead of a category, the review is by date, 
much in the manner of the calendar feature described above, 
rather than by category as also previously described. Other 
than this difference, the user interface is the same. 

Still other features and variations are possible in accor- 
dance with the invention. The common features of voice 
recognition and voice recording technologies provide for a 
more comfortable and familiar interface for the user. Such 
common features also allow for the implementations to be 
more comfortable in size, particularly in terms of making 
such implementations smaller. Such improvements over 
present technology allow for such personal information 
devices to be more personal and easy to use and, therefore, 
to better serve the primary purpose of this type of device. 
Consequently, such devices comprise easily carried and 
easily used means of storing and retrieving commonly 
needed information. 

While the invention has been particularly shown and 
described with reference to preferred embodiments thereof, 
it will be understood by those skilled in the art that various 
changes in form and details may be made therein without 
departing from the spirit and scope of the invention. 

What is claimed is: 

1. An electronic organizer comprising: 

a first transducer for converting acoustic sound energy 

representing spoken words into an electrical speech 

signal; 

a coupling circuit having an input for receiving the 
electrical speech signals and first and second outputs 
each for providing a respective output signal corre- 
sponding to the electrical speech signal; 

recording means coupled to said first output for recording 
an output signal in a form to permit reproduction of the 
corresponding electrical speech signals; 

word identifying means coupled to said second output for 
identifying a spoken word from among a selected 
plurality of spoken words; 

data storage means coupled to said word identifying 
means for storing data corresponding to each identified 
spoken word; 

function selection means including a plurality of manually 
operable function selection members; and 

control means coupled to said recording means, said word 
identifying means, said data storage means and said 
function selecting means for causing an output signal at 
said first output to be recorded by said recording means 
in response to manual operation of a first one of said 
function selection members, and for causing said word 
identifying means to identify, and said data storage 
means to store data corresponding to, at least one 
spoken word which is spoken following manual opera- 
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tion of a second one of said function selection mem- 
bers, said control means being responsive to sequential 
operations of the first and second ones of said function 
selection members for associating an output signal 
recorded by said recording means in response to the 5 
operation of the first one of said function selection 
members with the stored data corresponding to the at 
least one word spoken following the operation of the 
second one of said function selection members, 
wherein said coupling circuit is operative for subjecting 10 
the received electrical speech signals to a first process- 
ing that modifies the received electrical speech signals 
for speech recording and supplying signals which have 
undergone the first processing to said first output, and 
for subjecting received electrical speech signals to a 15 
second processing, different from the first processing, 
that modifies the received electrical speech signals for 
identification by said word identifying means and sup- 
plying signals which have undergone the second pro- 
cessing to the second output. 20 

2. An electronic organizer in accordance with claim 1 
wherein there are a plurality of types of data each associated 
with a group of selected spoken words that can be identified 
by said word identifying means, said function selection 
means are operable for selecting at least one of the types of 25 
data, said organizer further comprises a visual display hav- 
ing a plurality of display regions each associated with a 
respective one of the types of data, and said control means 
are coupled to said visual display for causing a readable 
representation of a spoken word identified by said word 30 
identifying means to be displayed in a display region asso- 
ciated with one of the types of data in response to operation 

of said function selection means to select the at least one of 
the types of data. 

3. An electronic organizer in accordance with claim 1 35 
wherein said control means are operative, in response to 
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selection of one of the types of data by said function 
selection means, for causing said visual display to produce 
a distinctive visual indication in the display region associ- 
ated with the selected one of the types of data, which visual 
indication informs a user of the type of data for which a word 
is to be spoken. 

4. An electronic organizer in accordance with claim 2 
when said control means are operative in response to opera- 
tion of said second one of said function selection members 
for causing said visual display to produce a distinctive visual 
indication in a first one of the display regions until said word 
identifying means has identified a spoken word associated 
with the same type of data as said first one of said display 
regions, and for causing said visual display to produce a 
distinctive visual indication in a second one of the display 
regions in response to identification by said word identifying 
means has identified a spoken word associated with the same 
type of data as said first one of said display regions. 

5. An electronic organizer in accordance with claim 1 
further comprising a second transducer for converting sig- 
nals recorded in said recording means into audible speech 
signals and a visual display for producing readable repre- 
sentations of data stored in said data storage means, and 
wherein said control means are coupled to said second 
transducer and to said visual display for outputting signals 
recorded in said recording means only to said second 
transducer and for outputting signals based on data stored in 
said data storage means only to said visual display. 

6. An electronic organizer in accordance with claim 1 
further comprising a casing containing said transducer, said 
coupling circuit and all of said means and shaped and 
dimensioned to be held in a user's hand. 

***** 
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the available field labels correspond respectively to different 
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DATA DISPLAY AND ENTRY USING A 
LIMITED-AREA DISPLAY PANEL 

TECHNICAL FIELD 

This invention relates to user interfaces for portable or 
handheld computing devices having limited display areas. 

BACKGROUND OF THE INVENTION 

Hand-held electronic organizers or PDAs (personal digital 
assistants) are becoming popular as replacements for paper- 
based organizers and appointment books. These devices are 
designed to be pocketable and easily accessible so that users 
feel comfortable taking them anywhere. They typically have 
miniature QWERTY keyboards and small user displays. 

Most electronic organizers include a variety of application 
programs such as address book, calendar, and task manage- 
ment application programs. While the computing capabili- 
ties of typical organizers are sufficient to implement fairly 
complex application programs, the small display screens 
present challenges and problems in designing convenient 
and effective user interfaces. 

FIG. 1 shows a portion of a user interface 29 that might 
be implemented by an address book program. Such a pro- 
gram typically maintains a plurality of records, correspond- 
ing respectively to different persons or businesses. Each 
record comprises a plurality of individual properties, such as 
first, middle, and last names, telephone numbers, address 
components, etc. 

The user interface of FIG. 1 includes three different types 
of field labels. Each field label is associated with a data entry 
field. The field label referenced by numeral 30 is of a type 
that initially occupies its associated data entry field. This 
type of field label is referred to herein as a "temporary" field 
label. The user can select the temporary field label and 
overwrite it with data. After data entry, the field label is no 
longer displayed. This type of field label is appropriate when 
a field type can be fairly easily identified by its data or by the 
location of the field in relation to other fields. 

The type of field label referenced by numeral 31 is 
positioned adjacent an associated data entry field 32 both 
before and after data is entered into the data entry field. This 
type of label, referred to herein as a "permanent" field label, 
is appropriate when data must be identified at all times by a 
label for the data to be meaningful. 

The field label referenced by numeral 33 is referred to 
herein as a "variable" field label. It can be changed by a user. 
It is associated with a data entry field 34. Like permanent 
field label 31, variable field label 33 remains visible both 
before and after data entry. However, a control 35 is posi- 
tioned adjacent the field label to allow a user to change it. 
Selecting the control or any part of the field label (for 
instance by touching it with stylus 43) activates or opens a 
pop-up list box 36 as shown in FIG. 2, containing a listing 
or menu of possible or available field labels. Pop-up boxes 
or controls such as these can be easily implemented in 
graphical operating system environments, where they are 
also referred to as combo boxes or drop-down boxes. The 
user can select one of the available field labels, whereupon 
the pop-up list box closes, as shown in FIG. 3, and the 
selected field label is displayed in the original location 
referenced by numeral 33. The field label actually selected 
and displayed on the user display is referred to herein as the 
"displayed" field label. 

A variable field label can be used when there is a limited 
display area. It is possible to use a single data entry field, 
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associated with a variable field label, for displaying and 
entering data relating to a plurality of individual properties. 
The user can select which property occupies such a data 
entry field at any particular time by selecting from the 

5 available field labels in the pop-up list associated with the 
variable field label. 

Unfortunately, user tests have shown that such a data 
display and entry scheme sometimes confuses users. This is 
because users often think that changing a variable field label 

30 will change only the label applied to a particular property — 
rather than the property associated with the data entry field. 
In the sequence of FIGS. 1, 2, and 3, for instance, field label 
33 initially reads "Car Tel," indicating that the correspond- 
ing data entry field 34 represents someone's car telephone 

15 number. When a user changes the field label to "Home Tel," 
the data previously entered for the "Car Tel" disappears from 
data entry field 34 because that data entry field is no longer 
used to represent the "Car Tel" property. Rather, it is now 
used to represent the "Home Tel" property, which initially 

20 contains no data. Although the data for "Car Tel" is still 
being stored, the user often does not understand this, and 
might believe that the previously entered data has been lost. 
Simply stated, the user often expects that the data in data 
entry field 34 will remain the same regardless of the field 

25 label chosen. 

This can seriously undermine the confidence of users. To 
meet the needs of the widest ranges of consumers, a user 
interface in a PDA should avoid any source of possible 
confusion while still allowing efficient and intuitive data 

30 display and entry. 

The invention described below reduces or eliminates the 
potential confusion resulting from the variable labeling 
scheme described above. This is accomplished without 

35 distracting display elements. The invention results in a user 
interface with greater clarity and usefulness than variable 
data field labeling schemes used in the prior art. 

SUMMARY OF THE INVENTION 

40 The invention allows a single data entry field to be used 
to display and accept data entry for a selected one of a 
plurality of properties. The data entry field is associated with 
a variable field label. The variable field label is associated 
with a pop-up list box containing a list of available field 

45 labels from which the user can select. In addition, each of the 
available field labels is followed by any existing data that 
has been entered for the corresponding property. This clari- 
fies to the user that data is being stored for each of the field 
labels shown in the pop-up list box. 

50 BRIEF DESCRIPTION OF THE DRAWINGS 

FIGS. 1-3 illustrate the operation of a prior art data 
display and entry interface. 

FIG. 4 shows a portable data entry device or electronic 
organizer accordance with the invention. 

FIG. 5 is a simplified block diagram of the electronic 
organizer of FIG. 4. 

FIGS. 6-9 show the operation of a data display and entry 
60 interface as implemented in accordance with the invention 
on the electronic organizer of FIG, 4. 

DETAILED DESCRIPTION OF THE 
PREFERRED EMBODIMENT 

65 FIG. 4 shows a hand-held electronic organizer, PDA 
(personal digital assistant), or other portable data entry 
device, generally designated by reference numeral 40. Orga- 
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nizer 40 has a miniature QWERTY keyboard 41 and a user 
display 42 comprising a small LCD screen or panel. The 
screen is preferably a touch-sensitive screen for use with a 
stylus 43. It has a preferred minimum resolution of 480x240 
pixels. In the preferred embodiment, the user display is a 5 
bit -mapped or pixel -addressable display, allowing the dis- 
play of both text and graphics. 

As shown in the block diagram of FIG. 5, organizer 40 has 
a data processor 44, memory 45, and I/O controllers 46 that 
are operatively connected to interact with the various hard- 10 
ware elements of the PDA such as keyboard 41. LCD screen 
42 is also responsive to data processor 44 through I/O 
controllers 46. Memory 45 includes battery-backed, 
randomly-addressable, read/write memory (RAM) for data 
storage, and read-only memory (ROM) containing an oper- 15 
ating system and pre-installed application programs. 

PDA 40 includes in its standard configuration a variety of 
application programs designed for execution by data pro- 
cessor 44. For instance, PDA 40 has address book, calendar, 
and task management application programs. These applica- 
tion programs operate under a graphical operating system 
similar to the Windows 95® operating system, produced by 
Microsoft Corporation of Redmond, Wash. The application 
programs use the graphical user interface features of the 
operating system to provide a familiar environment for users 
that might already be familiar with the Windows 95® 
operating system in a desktop environment. 

The invention is described below in conjunction with an 
address book program running from memory 45 on data 3Q 
processor 44, although the invention is also useful in other 
data display and entry contexts where display areas are 
limited. 

The read/write memory of electronic organizer 40 is 
accessible by data processor 44 for storing and retrieving 3S 
data records such as an address book application program 
might maintain for different persons. Each record includes a 
plurality of individual stored properties. For instance, an 
address book record might include name, address, and 
telephone number properties. Each record potentially has 40 
more properties than can be meaningfully displayed simul- 
taneously on LCD screen 42. In other words, the LCD screen 
has a display area that is too small to simultaneously display 
all the individual properties of a data record in a way that is 
easily understandable by a user. 45 

FIG. 6 shows a portion of preferred user interface 60 that 
might be used for data display and entry in an address book 
program on a device such as electronic organizer 40 that has 
a limited display area. User interface 60 comprises three 
different types of fields labels as discussed above in the 50 
Background section of this disclosure. The first type of field 
label, referred to herein as a "temporary" field label, is 
indicated by reference numeral 61. This type of field label 
initially occupies the same display area that forms the data 
entry field corresponding to the field label. When data is 55 
entered in the data entry field, the field label is removed. For 
instance, the data entry field designated in FIG. 6 by 
reference numeral 61 initially contained a field label reading 
"Name". Once actual data (in this case the name "Robert 
Bell") was entered into the data entry field, however, the go 
field label disappeared and was replaced by the data. The 
field name would reappear if the data were to be subse- 
quently deleted from the data entry field for the particular 
record being displayed. 

To enter data corresponding to a temporary field label, the 65 
field label is first selected by a user. The user selects a field 
label by touching it with a stylus, clicking on it with a 



mouse, or by moving a cursor to it with keyboard controls. 
The field label indicated by reference numeral 63 has been 
selected by a user. When selected, the entire field label is 
highlighted (indicated by a box for purposes of illustration). 
In contrast to prior art user interfaces of this nature, the 
entire field name remains visible until the user actually 
begins data entry. When the user presses the first data entry 
key, the entire field label disappears and is replaced by 
whatever data is being entered. 

An example of a second type of field label, referred to 
herein as a "permanent" field label, is indicated in FIG. 6 by 
reference numeral 65. A permanent field label is perma- 
nently displayed and is permanently associated with a sepa- 
rate data entry field. Field label 65 is associated with an 
adjacent data entry field 66. To enter data corresponding to 
a particular field label, the user selects the adjacent data 
entry field and simply begins entering data. The data entry 
field initially contains some sort of indication that it is 
empty, such as the ellipsis referenced by numeral 67 of FIG. 
6. The ellipsis is deleted from the data entry field when data 
is entered into the field. 

Two examples of the third type of field label are refer- 
enced by numerals 70 and 75 in FIG. 6. This type of field 
label is referred to herein as a "variable" field label. It is very 
similar to a permanent field label, in that it is associated with 
an adjacent data entry field, in this case with data entry fields 
71 and 76, respectively. The data entry field is selectable by 
the user to enter a value for the stored data field or property 
corresponding to the variable field label. A field label such 
as field label 70 is not replaced by entered data. The data is 
entered on a separate data entry field. 

A variable field label differs from a permanent field label 
in that it can be changed by the user. In FIG. 6, field label 
70 has been set to read "Mobile Tel". Corresponding data 
has been entered in associated data entry field 71. The user, 
however, can change field label 70 to text that corresponds 
to a different record property. This is illustrated in FIG. 7, 
which symbolically shows a portion of a database record 90. 
The database record contains a plurality of fields, each 
referenced by a field label shown in the first column of the 
record. Each field also potentially contains data, shown in 
the second column of the record. The first field, for example, 
has a field label "Mobile Tel" and corresponding data 
"555-4444". Fields which do not yet contain data, such as 
the "Internet" field, are indicated by an ellipsis in the second 
column. 

The portion of database record 90 is not displayed as 
illustrated in FIG. 7 to the user. Rather, a user is able to select 
any of these field labels as one of variable field labels 70 or 
75. When a particular field label is chosen, the associated 
data appears in data entry field 71 or 76. 

The process of selecting one of the field labels for variable 
field label 75 is shown in FIGS. 8-9. Label 75 initially reads 
"Other"(FIG. 6), indicating that the user has not yet made a 
selection for this label. No actual data is displayed or 
allowed to be entered in data entry field 76 when "Other" is 
the displayed field label. 

As illustrated in FIG. 8, a pop-up list is associated with 
displayed field label 75. The pop-up list is selectively 
displayed in response to a user's selection. In the preferred 
embodiment, user interface 60 includes a control 77 asso- 
ciated with field label 75. The pop-up list is displayed or 
activated when the user selects control 77. 

FIG. 8 shows user interface 60 after a pop-up fist box 78, 
associated with field label 75, has been activated. Pop-up list 
box 78 contains a plurality. of available field labels 79, 
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corresponding to the fields of database record 90 of FIG. 7, 
that can be alternatively selected by the user. When an 
available field label is selected, it becomes the displayed 
field label 75. Each available field label 79 corresponds to a 
different stored record property or data field, as explained 
with reference to FIG. 7. When a particular field label is 
selected to be the displayed field label, data entry field 76 
displays data for the corresponding property, and allows data 
to be entered or modified for the corresponding property. 
When the displayed variable field label 75 is subsequently 
changed by the user, data entry field 76 displays and allows 
editing of data for a different property, corresponding to the 
newly selected field label. 

In addition to a listing of available field labels, pop-up list 
box 78 contains an indication of any existing values for the 
stored properties corresponding to the available field labels. 
More specifically, each available .field label is followed by a 
textual representation 80 of any data that has already been 
entered for the corresponding property. For instance, the 
available field label "Home Tel" is followed by the actual 
data already contained in the "Home Tel" record property 
(555-5555). In the preferred embodiment, the available field 
labels are positioned within the pop-up list box adjacent 
their corresponding values. For those properties for which 
values have not been entered, the pop-up list box includes an 
empty indicator, as referenced by numeral 81 in FIG. 7, 
indicating that a value has not yet been entered. 

FIG. 9 shows user interface 80 after "Home Tel" has been 
selected as the displayed field label. The user selects this 
label from list box 78 by touching it with the stylus or by 
using the keyboard. After the selection, data entry field 72 
contains the data corresponding to the displayed field label, 
in this case the field labeled "Home Tel". 

At any time, the user can re-activate the pop-up list box 
to again change variable field label 75. The same selections 
are available to the user, and the user can see the data 
previously entered for any of the available database fields or 
properties. 

While the invention has thus far been described primarily 
in terms of structural or graphical features associated with a 
user display, the invention also includes methodological 
steps that are preferably performed by the data processor or 
other logic components of a portable data entry device. 
These steps include placing or displaying a displayed field 
label on a user display, and placing or displaying a data entry 
field on the user display in proximity to the displayed field 
label. The data entry field contains an indication of any 
existing value for a stored property corresponding to the 
displayed field label. The invention further includes a step of 
associating a pop-up list box with the displayed field label, 
containing a plurality of available field labels. The available 
field labels correspond respectively to different stored prop- 
erties. 

Further steps in accordance with the invention include 
selectively displaying the pop-up list box on the user display 
in response to a user's selection, and selecting one of the 
available field labels in the pop-up list box as the displayed 
field label in response to user input. The invention also 
includes a step of including in the pop-up list box an 
indication of any existing values for the stored properties 
corresponding to the available field labels. Such an indica- 
tion clarifies to the user that the available field labels 
correspond respectively to different stored properties. Empty 
indicators are included in the pop -up list box, associated 
with any available field label having a corresponding stored 
property for which a value has not been entered. 
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Indicating the data, within the pop-up list box, for prop- 
erties for which data has already been entered clarifies to 
users that the available field labels correspond respectively 
to different stored properties rather than just being altcma- 
5 tive labels for the same property. It has been found that this 
scheme greatly reduces or even eliminates user confusion 
regarding this issue. 

In compliance with the statute, the invention has been 
described in language more or less specific as to structural 
30 and methodological features. It is to be understood, 
however, that the invention is not limited to the specific 
features and steps described, since the means herein dis- 
closed comprise preferred forms of putting the invention 
into effect. The invention is, therefore, claimed in any of its 
]5 forms or modifications within the proper scope of the 
appended claims appropriately interpreted in accordance 
with the doctrine of equivalents. 

We claim: 

1. A portable user interface comprising: 
20 a user display having a display area that is too small to 
simultaneously display all of the individual properties 
of a data record in a way that is easily understandable 
by a user; 

25 a plurality of displayed field labels on the user display; 
a pop-up list box associated with a particular one of the 
displayed field labels, the pop-up list box being selec- 
tively overlaid on the displayed field labels in response 
to a user's selection; 
30 the pop-up list box containing a plurality of available field 
labels that can be alternatively selected by the user as 
the displayed field label, wherein the field labels cor- 
respond respectively to different stored properties; 
a data entry field corresponding to said particular one of 
35 the displayed field labels, the data entry field being 
displayed on the user display simultaneously with the 
plurality of displayed field labels, the data entry field 
being selectable by a user to enter a value for the stored 
property corresponding to the displayed field label; 
40 the pop-up list box further containing an indication of any 
existing values for the stored properties corresponding 
to the available field labels, said indication clarifying to 
the user that the available field labels correspond 
respectively to different stored properties. 
45 2. A user interface as recited in claim 1 wherein the 
pop-up list box includes an empty indicator associated with 
any available field label having a corresponding stored 
property for which a value has not been entered. 

3. A user interface as recited in claim 1 wherein the 
50 respective available field labels are positioned within the 

pop-up list box adjacent the indications of existing values 
for the corresponding stored properties. 

4. A portable data entry device, comprising: 
55 a data processor; 

data memory accessible by the data processor for storing 
data records, each data record including a plurality of 
individual stored properties; 

a user display that is responsive to the data processor, the 
60 user display having a display area that is too small to 
simultaneously display all of the individual properties 
of a data record in a way that is easily understandable 
by a user; 

the data processor being programmed to display indi- 
65 vidual stored properties of a data record by performing 
the following steps: 

placing a displayed field label on the user display; 
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associating a pop-up list box with the displayed field 
label, the pop-up list box containing a plurality of 
available field labels, wherein the available field 
labels correspond respectively to different stored 
properties; 5 

selectively overlaying the pop-up list box on the user 
display containing the displayed field label in 
response to a user's selection; 

selecting one of the available field labels in the pop-up 
list box as the displayed field label in response to 10 
user input; 

displaying a data entry field on the user display simul- 
taneously with the displayed field label, the data 
entry field containing an indication of any existing 
value for the stored property corresponding to the is 
displayed field label; 

including in the pop-up list box an indication of any 
existing values for the stored properties correspond- 
ing to the available field labels, said indication 
clarifying to the user that the available field labels 20 
correspond respectively to different stored proper- 
ties. 

5. A portable data entry device as recited in claim 4 
wherein the data processor is further programmed to per- 
form a step of including in the pop-up list box an empty 25 
indicator associated with any available field label having a 
corresponding stored property for which a value has not 
been entered. 

6. A portable data entry device as recited in claim 4 
wherein the data processor is further programmed to per- 30 
form a step of positioning the respective available field 
labels within the pop-up list box adjacent the indications of 
existing values for the corresponding stored properties. 

7. A method of displaying stored data properties on a 
portable user interface, comprising the following steps: 35 

placing a displayed field label on a user display, the user 
display having a display area that is too small to 
simultaneously display all of the individual properties 
of a data record in a way that is easily understandable 
by a user; 40 

associating a pop-up list box with the displayed field 
label, the pop-up list box containing a plurality of 
available field labels, wherein the available field labels 
correspond respectively to different stored properties; 

selectively overlaying the pop-up list box on the user 
display containing the displayed field label in response 
to a user's selection; 

selecting one of the available field labels in the pop-up list 
box as the displayed field label in response to user 50 
input; 

displaying a data entry field on the user display simulta- 
neously with the plurality of displayed field labels in 
proximity to the displayed field label, the data entry 
field containing an indication of any existing value for 55 
the stored property corresponding to the displayed field 
label; 

including in the pop-up list box an indication of any 
existing values for the stored properties corresponding 
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to the available field labels, said indication clarifying to 
the user that the available field labels correspond 
respectively to different stored properties. 

8. A method as recited in claim 7 and further comprising 
a step of including in the pop-up list box an empty indicator 
associated with any available field label having a corre- 
sponding stored property for which a value has not been 
entered. 

9. A portable data entry device as recited in claim 7 and 
further comprising a step of positioning the respective 
available field labels within the pop-up list box adjacent the 
indications of existing values for the corresponding stored 
properties. 

10. A computer-readable storage medium containing 
instructions that are executable by a computer to perform 
steps comprising: 

placing a plurality of displayed field labels on a portable 
user display having a display area that is too small to 
simultaneously display all of the individual properties 
of a data record in a way that is easily understandable 
by a user; 

associating a pop -up list box with a particular one of the 
displayed field labels, the pop-up list box containing a 
plurality of available field labels, wherein the available 
field labels correspond respectively to different stored 
properties; 

selectively overlaying the pop-up list box on the user 
display containing the displayed field labels in response 
to a user's selection; 

selecting one of the available field labels in the pop-up list 
box as the particular one of the displayed field labels in 
response to user input; 

displaying the selected field label as the particular one of 
the displayed field labels; 

displaying a data entry field on the user display simulta- 
neously with the plurality of displayed field labels in 
proximity to the displayed field label, the data entry 
field containing an indication of any existing value for 
the stored property corresponding to the displayed field 
label; 

including in the pop -up list box an indication of any 
existing values for the stored properties corresponding 
to the available field labels, said indication clarifying to 
the user that the available field labels correspond 
respectively to different stored properties. 

11. A computer-readable storage medium as recited in 
claim 10, the instructions being executable to perform a 
further step of including in the pop-up list box an empty 
indicator associated with any available field label having a 
corresponding stored property for which a value has not 
been entered. 

12. A computer-readable storage medium as recited in 
claim 10, the instructions being executable to perform a 
further step of positioning the respective available field 
labels within the pop-up list box adjacent the indications of 
existing values for the corresponding stored properties. 
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ABSTRACT 



A system for facilitating the searching of data from an 
electronic data source. The invention teaches a method for 
presenting a large number of search terms rapidly and 
efficiently to a user. Users may scan a large number of search 
terms, move quickly from one search category to another, 
select a search term with minimal mechanical input and 
generate a series of "hits" that meet their search goals. The 
need to guide the computer with mechanical actions from a 
keyboard, mouse or other device is minimized. Graphics that 
distract the user from any task other than selecting hits are 
eliminated. Users are able to scan quickly and find electronic 
information with minimum effort. 

32 Claims, 9 Drawing Sheets 
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SEARCH SYSTEM HAVING USER- 
INTERFACE FOR SEARCHING ONLINE 
INFORMATION 

BACKGROUND OF THE INVENTION 

Information is available in quantities significantly larger 
than has ever been known in human history. For example, 
EXCITE, one popular search engine on the World Wide 
Web, now claims to routinely examine and index 250 
million pages of online information and to consider another 
500 million pages for possible inclusion in the EXCITE 
search system. Information is also available to more people 
than ever before in human history. The use of electronic 
information has increased more than a thousand fold since 
1990. 

Interface technology that assists searchers in locating 
target information from all available information is still 
evolving. One popular method for finding information 
online is a Key Word search. A user types a word or words 
in a search box provided for this purpose. The word(s) typed 
are used to search all available documents and return "hits" 
to the user. Most modern online electronic search engines, 
including well known search sites such as ALTAVISTA and 
LYCOS, include Key Word searching. 

A second popular method for finding information is by 
presenting a Directory. Typically, Directory searching 
involves presenting a list of categories. The user selects from 
among the choices that are displayed. Additional, more 
specific, search terms are then presented and selected by the 
user. The search process is organized hierarchically so that 
a selection from one category leads to a set of other choices 
contained within that category. A user moves down the 
"branches" of the directory "tree" until they find the infor- 
mation they want. YAHOO and MICROSOFT NETWORK 
are examples of the many publicly available electronic 
search engines that provide Directory listings. 

Cochran describes non-hierarchical searching in U.S. Pat. 
No. 5,768,581, which is herein incorporated by reference. 
Instead of moving down a Directory tree, a user selects 
search terms from several search categories that are not 
organized in a branching tree structure. Selecting from one 
category does not eliminate the possibility of selecting from 
other categories because categories are mutually exclusive, 
unlike Directory searching. 

Cochran et al., in U.S. Pat. Nos. 4,879,648 and 5,206,949 
herein incorporated by reference, describe two distinct ways 
of developing lists of search terms. Static lists are developed 
"logically" from what is already known about how infor- 
mation is organized and used. Static lists are constructed 
from theory and/or the expected use of the information and 
are prepared in advance of use of the lists. Static lists do not 
require a search through the information source, and remain 
the same unless a list of terms in the program itself is edited. 
For instance, a static list may be an alphabetical list of city 
names in a company's service area. It is relatively unchang- 
ing and can be constructed by management personnel in 
advance of any search. 

Dynamic lists, on the other hand, are developed by an 
active search of the information under consideration. 
Dynamic lists vary over time as the information source 
changes, and may change frequently. For instance, a 
dynamic list may be an alphabetical list of the names of 
people who have enrolled in an online promotion. The list 
changes from minute to minute as people sign up. r ITie 
Cochran patents describe one way of constructing static and 
dynamic lists. 
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All online information search systems that display select- 
able is search terms are relatively slow. It often takes more 
than a minute to select multiple search terms from a Direc- 
tory or non-hierarchical search system. A user must click 

5 repeatedly to scroll lists, to move down a page or to change 
pages in order to find the information they are looking for. 
People have the capacity to look at choices much faster than 
the choices are presented by these displays. As more users 
search more and larger sources of online information, there 

3Q is a tremendous loss of time waiting for computer displays 
to catch up with the human ability to scan and select. 

The relative slowness of presenting search terms is not a 
function of search speed or communication speed. Computer 
hardware and communication bandwidth is now sufficient 
for quick interactions. The slowness of search technology is 

15 caused mainly by inadequate user-interface technology. 
A method for rapidly locating pages on a web site is 
shown, for instance at www.sherwin.com, which is operated 
by The Sherwin-Williams Co. The site describes products 
and services offered by The Sherwin-Williams Company. 

20 One page, Products and Services, at www.sherwin.com/ 
productsservices/default.asp, shows several menus and sub- 
menus that function as hyperlinks to other pages within the 
Sherwin-Williams website. Sub-menus that are not initially 
visible are displayed when a cursor is moved over a dis- 

25 played menu items, which is referred to as a "mouse over". 
The display makes hyperlinks within it's website available 
quickly, in a relatively small amount of space. 

However, the Sherwin-Williams website is not a search 
site and is not used to search a database or other information 

30 source. The site provides hyperlinks to individual pages 
within the Sherwin-Williams website. When a user has made 
one choice on sherwin.com/productsservices/default.asp, a 
different page is displayed and no additional choices are 
possible. 

35 With the Sherwin-Williams technology, a user cannot 
access information from a database of information. A user 
cannot make successive choices to refine their selection as 
they would with a Directory or non-hierarchical search. 
When the user selects any hyperlink, the user is then 

40 presented with a different page. The different page does not 
offer any menu choices as on the Services and Products 
page. There is no database search function associated with 
the display on the Sherwin-Williams site. 

Accordingly, there is a need to facilitate and speed up the 

45 presentation and selection of search technologies. This, in 
turn, will substantially increase the commercial value and 
practicability of online searching. 

SUMMARY OF THE INVENTION 

50 It is an object of the present invention to reduce the 
amount of time needed to display and select search terms 
from any user-interface that displays search terms. It is an 
object of the invention to eliminate or minimize features that 
add unneeded complexity to a display. This includes elimi- 

55 nating the use of scroll bars, task bars and graphics that must 
be clicked to signal page reorganization or page changes. It 
is also an object of the present invention to eliminate the 
need for any mechanical activity on the part of the user that 
interferes with the users ability to scan large numbers of 

60 search terms and select search terms quickly. 

Users may scan and select quickly without the need to 
interpret a variety of icons and instructions and without the 
need to perform more than the minimum number of physical 
actions. The user is not slowed down by pick boxes, scroll - 

65 ing pages, scrolling lists of hyperlinked terms or selecting 
functions from a task bar, as required by conventional 
systems. 
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These goals are achieved by presenting search terms that to make it easier to search and retrieve information. When 

are meaningfully organized and then advancing the display there are large quantities of information, the information is 

of choices when a mouse or other pointing device passes preferably stored on a server. 

over specific parts of the display screen. The display changes The system and method of the present invention is imple- 

quickly to reveal different or additional search terms. If the 5 mented by computer software that permits the accessing of 

user wants a list that is displayed to be removed from the data from an electronic information source. The software 

visual display, the user simply moves the pointing device to and the information in accordance with the invention may be 

a "blank" area of the display screen. within a single, free-standing computer or it may be in a 

The present invention reduces, and almost eliminates, the central computer networked to a group of other computers or 

clicking, scrolling and page reorganization that has been 30 other electronic devices. The information may be stored on 

required by search directories and non-hierarchical search a computer hard drive, on a CD ROM disk or on any other 

systems heretofore. Hence, mechanical actions and unnec- appropriate data storage device. 

essary decision making that slow the selection of search FIG. 1 shows a simplified representation of a preferred 

terms are eliminated. The display of graphics is simplified so overall implementation of the invention in a computer 

the user's attention is not distracted from the process of 35 network 100 in accordance with the preferred embodiment, 

scanning and quickly selecting search terms. A plurality of client computers 120 are networked to a 

remotely located server 110 by a communication link 150 

BRIEF DESCRIPTION OF THE DRAWINGS and bidirectional communication lines 130. 

FIG. 1 shows a general configuration of a computer Client computers 120 contain, at a minimum, memory 

network used in accordance with the present invention. 20 121, processing capability 123, a display device 125 and a 

FIGS. 2a-2c are flow charts showing the decision rules P ointm 8 device 127 ■ ^ xrvet contains at a minimum, 

for implementing the invention. *«.° ra S e ca P acltv 112 - memory 116 and processing capability 

T ^„, _ , . /.,. 11"- As a practical matter, the server also has a display 

FIG. 3 shows a preferred embodiment of the invention, an d6vic6 n? and a fc board (n0 , shown) 

online database tor searching a lodging database. 25 ™ ^ . t . T . . , tl _ . 

to to & The Communication Link 150 and the communication 

FIGS. 4-6 show the displays of three lists from the Unes 13Q provide (wo way commun i cat ion between clients 

lodging database. 120 an d the server llO.The link is established when a client 

FIG. 7 shows the selection on one of the displayed terms. 120 accesses the server at its electronic address 118. This is 

FIG. 8 shows the effect of the selection of the term in FIG. 3Q done, for example, by entering the Internet address of the 

7. server 118 using any Web Browser software. 

FIG. 9 shows an alternative preferred embodiment of the Programing language may be imbedded in the Browser to 

invention, an Internet Search Engine. allow some of the processing required by the invention to be 

FIG. 10 shows the display of Category Names on the done on me client rather than having all of the programming 

Search Engine 35 on server. JavaScript, which was developed by 

m _ , . . „ < , , Netscape Communications Corporation, is a well known 

FIG .11 shows the position of the cursor when selecting language use(J {Q embed programming ^ int0 html docu . 

a term from a Category. ments Jn addition ^ memory space 116 is optionally allocated 

FIG. 12 shows a sub Category list. ^ tnat server 110 may retain the status of search requests 

FIG. 13 shows search terms that are displayed after 4Q generated by individual computers 120 during any indi- 

selecting a sub-Category. vidual search session. 

FIGS. 2a-2c are a generalized overall method of finding 
information from an electronic information source in accor- 
dance with the preferred embodiment of the invention. For 

In describing a preferred embodiment of the invention, 45 illustrative purposes specific examples will be discussed 

specific terminology will be resorted to for the sake of with respect to FIGS. 3-12. 

clarity. However, the invention is not intended to be limited Category Names can be generated as either static or 

to the specific terms so selected, and it is understood that dynamic lists 10. If they are dynamic ("yes" at 10), a search 

each specific term includes all technical equivalents that is made of the information source and a list of search terms 

operate in a similar manner to accomplish a similar purpose. 50 is generated 11. If the first list of Category Names exists as 

As user herein, the phrase, "search term" generally refers a static list ("no" at 10), the Category Names are identified 

to any word, symbol or phrase that can be selected by a user. 12. 

Search terms are used to search electronic information for Initial lists of Search Terms for each category may be 

the purpose of identifying and locating parts of information static or dynamic 13. If they are dynamic, a search is made 

of interest to the user. The word "category" generally is used 55 of the information source and a list of terms is generated 14. 

to refer to a group of search terms that are logically related. IF the first list of search terms exist as a static list, the search 

In some situations, the category name itself may also be a term lists are identified 16. Category Names and lists of 

search term. Search Terms associated with each Category Name are sent 

A display device is a computer, television terminal, hand- to the client 18 along with graphics and other information 

held electronic device or any implement that allows a user 60 needed to display the page on which the search for infor- 

to interact with a visual representation of an information mation is to take place. Embedded programming code may 

source. A pointing device is a cursor, track ball, touch pad, also be sent to the client. 

pen light or other implement used to interact with iconic Continuing with FIGS. 2b and 2c, Category names are 

images and information displayed on a display device. An displayed with graphics and other information 20. Search 

information source may be original information, in the form 65 Terms are displayed when the pointing device passes over a 

of a database or other collection of records, or it may be a Category Name 22. The search terms are displayed quickly, 

secondary source that reorganizes original content in order using "mouse overs" to test the position of the pointing 
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device. That is, when the pointing device is passed over a the server, steps 11, 12, and Search Term lists. As described 

Category Name, the list of search terms appears on the below, some lists are static and some dynamic, 13, 14, 16. 

screen "automatically'*. It is not necessary to activate the The Category Names and Search Term lists are transmitted 

pointing device by, for example, clicking a cursor in order to to the client 18 and the Category Names are displayed 20 as 

display the list of search terms. Lists change when the 5 shown in FIG. 3. A logo 210, as well as directions 212 of 

pointing device is moved to cover any Category name. If the how to proceed, may optionally be displayed, 

pointing device is not over a Category Name, no list is The number of "hits" or listings in Maryland from which 

displayed. to choose 214 are displayed. Here, the total number of 

To select a term and proceed ("yes" at 24), it is necessary listings for Maryland is 402, as indicated. Nine Category 

to activate the pointing device 24. If a cursor is being used, *o Names appear in two columns 216-232. In this embodiment, 

the left button of a cursor is clicked on one of the terms to 218, 220 and 222 have lists that are static. Each search term 

select that term. Activating the pointing device results in a that appears in the list is known in advance of the search 

search of the information source 25. There may be no more since the search terms are always the same. The six other 

search terms from which to choose ("No" at 26) and the lists are dynamic, and thus will be different for New York 

program is finished 28. is and other states than they are for Maryland. They are 

However, in more complex situations, additional choices obtained by searching the information source, 
are possible ("Yes" at 26). If the search design is The non- hierarchical search system at Lot'a Places is 
hierarchical, as is the case with a Directory, Category Names organized so that any of the nine categories can be used to 
are likely to change ("Yes" at 32), because, by definition, reduce the number of hits from 402 to some smaller number. 
Directories narrow the range of choices by moving to 20 Near the bottom of the display, the beginning of an alpha- 
smaller and smaller divisions of the domain. Those divisions betical list of the current hits is displayed 234. (The inven- 
have different sub-category names. If the design is non- tion does not require hits 234 to be displayed as searching 
hierarchical, Category Names are not likely to change ("No" progresses. As an alternative, hits may only be displayed 
at 32) because in a non-hierarchical search, the search terms when requested by the user.) The cursor 200 is on an 
are reduced without necessarily changing the categories. 25 "empty" part of the screen so that no lists of search terms are 

If Category Names change ("Yes" at 32), a new list of displayed; 

Category Names is generated. At step 34 the program FIG. 4 shows the same display when the cursor 200 is 

determines whether the Category Names are at the server, or passed over the Category Name, Features 216. A list of 

the client and whether they are static or dynamic 35, 37. If search terms for the category, Features, is displayed 236, 

they are dynamic, the new list of Category Names is then step 22. The search terms are displayed "over** or "on top of 

generated at the server 38 or the client 36. If the Category the original html page. Variations in page color or texture 

Names are generated at the server, they are transmitted to the can be used to indicate that some material rests "on top of 

client 40. If Category Names do not change ("No" at 32) the other material. 

program examines the status of the search terms 42 and If the user moves the cursor to some "empty" place on the 

proceeds. display, the list disappears. The user can then put the cursor 

Lists of Search Terms may not change ("No" at 42), in over another category 218-232 and see another list, 

which case the user may select again 24. However, lists of Alternatively, the user may click on one of the search terms 

Search Terms often do change ("Yes" at 42). New lists of in the list 24 and execute a search, step 25. 

Terms may be generated on the sever ("Yes" at 44) or the 4Q FIG. 5 shows a second list of search terms 238. This list 

client ("No" at 44). At steps 46 and 50, the system deter- is displayed when the cursor 200 is passed over the category, 

mines whether the Search Term lists are dynamic or static. Lodging Name 224. The list is removed if the user moves the 

If lists are dynamic, a search will be required 48, 54. If the cursor off the category name. FIG. 5 also shows the word 

new Terms are generated on the server, they are transmitted "more" in the last position of the second column 240. If the 

to the client 52. All list of new terms are ultimately available 45 user passes the cursor over this term, additional search terms 

to the user 22. in the same category are displayed. The use of the "more" 

This design provides maximum flexibility for search option makes large numbers of choices available in a small 

situations. The system may be used with hierarchical Direc- amount of space at a fastest possible speed. The "more" 

tories and non-hierarchical search systems. The system may option may be used multiple times within a list to display 

also be implemented as a server-based system, as a client- 50 multiple pages. 

based system or as a combination of both, FIG. 6 shows a list of Search Terms 242 that is displayed 

Users can manipulate the search process multiple times, when the cursor 200 is over Price Category 220. FIG. 7 

very quickly, until the desired information is identified. shows the position of the cursor 200 on the list of search 

Once a term is available ("Yes" at 26), every path in the flow terms 242 when it is used to select the search term, Upscale, 

chart returns to display Category Names and/or Lists 20, 22. 55 from the Price Category 220. 

Terms will be displayed rapidly in a small amount of space FIG. 8 shows the results of selecting Upscale at step 24 in 

without repositioning the screen display. FIG. 7. A number of changes have taken place. A search of 

FIG. 3 shows a preferred embodiment of the current the database was conducted when the term, Upscale was 

invention, a non-hierarchical search system for finding lodg- selected, step 25. Instead of there being 402 hits in 

ing locations in the United States. FIG. 3 is a second display 60 Maryland, there are now seventy-four hits, as shown at 214. 

at a web site named Lot'a Places (www.lotaplaces.com). The seventy-four hits that were selected correspond to 

Prior to seeing the display in FIG. 3, a user selected from a lodging in Maryland rated with an upscale price (rather than 

page that presented choices about the location of the lodg- Mid-range or Economy price, for example). The alphabeti- 

ing. cal list of hits has changed 234 to show only upscale 

In the example that is used here, the state of Maryland was 65 properties, 

selected on the prior page. Once a State is selected, the At this point, the system determines if additional search 

preferred embodiment identifies static Category Names on terms are available amongst the Categories, step 26 and if 
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the Category Names, have changed 32. In this example, Accordingly, non-hierarchical designs (FIGS. 3-8) and 

Price Category 220 has been muted to show that the category Directories (FIGS. 9-13) have different advantages and 

is no longer available for selection during this search. In the disadvantages. Yet, both operate more quickly with this new 

present example, search terms are available in each of the technology. The preferred embodiment uses a Java applet to 

remaining (non-muted) Categories 216-218, 222-232. s manage the online display. The applet sets the maximum 

When the cursor 200 is passed over the Category Name, number of rows to be displayed in each column of search 

Lodging Name 224, the list 244 is considerably shorter than terms. It also sets the maximum number of columns and 

the list of Lodging Names displayed in FIG. 5. Lodging column width. 

Names are now only the names in Maryland that are rated The display technique of the present invention may also 

with an Upscale price. When the search was conducted as be implemented for e-commerce purchasing and corporate 

illustrated in FIG. 7, the domain under consideration was information systems. In addition, the system may be used in 

reduced from 402 hits to 74 hits and all dynamic lists of a local computer environment, such as searching files in 

search terms were updated. local storage. 

A distinguishing feature of this invention is the quickness It is not desired to limit the invention to the specific 

and fluidity of the selection process. This quickness and JS examples shown and described. Rather, all suitable modifi- 

fluidity can be achieved even when space needed for lists of cations and equivalents may be resorted to, falling within the 

search terms exceeds the display space of the computer scope of the invention, 

screen. There are no scroll bars and no pick boxes that I claim: 

change the display of search terms on this single, stable L A method for searching an electronic information 

page. 20 source using a display device and a pointing device dis- 

"Mouse overs" allows a list to be displayed when the played on the display device, the method comprising: 

cursor is passed over a Category Name, and is used in the displaying search category names on the display device; 

present invention to obtain a quick and stable selection enabling the user to display a list of search terms asso- 

process. In addition, search terms are passed to the client or ciated with a category name by passing the pointing 

created on the client, and held in readiness on the client until 25 device over any of the displayed category names; and 

the user passes the cursor over a Category Name to display enabling the user to select one of the displayed search 

a list of Terms. Actual searches of an information source terms. 

such as a lodging database are planned so that new and 2 . The method of claim 1, wherein the pointing device 

additional information will flow to the user at appropriate comprises a mouse having a button, and the step of enabling 

times during the selection process. The downloading of lists 30 tne user t0 display a list of search terms does not require the 

and programming code is balanced to minimize delays uscr t0 0 p era te the button. 

experienced by users. 3 The me thod of claim 1, further comprising enabling the 

FIGS. 3-8, for the Lot'a Places Lodging search, illustrate use r to display a different list of search terms by passing the 

the current invention applied to a non-hierarchical search pointing device over a different category name, 

system. FIGS. 9-13, illustrate the use of the current inven- 35 4. The method of claim 1, further comprising enabling the 

tion when the search system is a hierarchical Directory. FIG. user to clear the displayed search terms by moving the 

9 shows the first page of an Internet Search Service. A title pointing device away from the displayed search terms and 

246 appears at the top of the page. Category Names 248 selected category. 

preferably are displayed in a single column on the left side 5. The method of claim 1, wherein at least one list of 

of the page. The cursor 200 is not covering any Category 40 search terms is updated in response to the user selecting one 

Names. Therefore, no lists of search terms are displayed. 0 f the displayed search terms. 

FIG. 10 shows the list of search terms 250 that are 6. The method of claim 1, wherein the information source 

displayed when the cursor 200 is passed over the name, is located remote from the display device. 

Business & Economy 252, Search terms from any of the 7. The method of claim 1, further comprising searching 

other Category Names will be displayed in the same way if 45 the information source and retrieving information from the 

the cursor is passed over those names. Lists are all available information source in response to said selecting, 

on the client and can be displayed by moving the cursor to 8. The method of claim 1, further comprising displaying 

the appropriate position. FIG. 11 shows the position of the on the display device an indicator indicating that additional 

cursor 200 as the user selects Business Opportunity 254, the search terms are available that are not displayed on the 

first Search Term in the list. 50 display device, and enabling the user to display the addi- 

FIG. 12 shows the result of clicking on the Search Term, tional search terms by passing the pointing device over the 

Business Opportunity 254 in FIG. 11. A visual display of the indicator. 

Category and sub-category that was selected is presented at 9. The method of claim 1, wherein at least one category 

step 256. The search returns another list of Categories 258, name changes in response to said selecting, 

which is displayed in place of the list of Category Names in 55 10. The method of claim 1, wherein at least one list of 

FIGS. 9, 10 and 11. As described in connection with the flow search terms is dynamically generated by a search of the 

chart, FIGS. 2a and 2b, Category Names sometimes change. information source. 

In particular, Directories, like the Internet Search Engine in 11. The method of claim 9, wherein the category names 

this figure, are likely to create subdirectories of additional are arranged in a hierarchical manner. 

Categories as the search process moves down the branches 60 12. The method of claim 1, wherein successive selections 

of a Directory tree. In this case, the result of selecting reduce available search terms. 

Business Opportunities 254 is a subdirectory that gives 13. The method of claim 1, further comprising searching 

names of Business Opportunities. Passing the cursor over the information source in response to said selecting, 

these terms may result in other subcategories or it may 14. The method of claim 13, wherein said searching is 

display a list of hits. FIG. 13 shows the result of passing the 65 conducted at a client processor. 

cursor over Investment Opportunities in the Business 15. The method of claim 1, wherein at least one category 

Opportunities sub-category. name is updated in response to the selecting. 
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16. The method of claim 1, wherein a search term 
comprises any word, symbol or phrase used to search 
information stored in the electronic information source. 

17. The method of claim 1, the electronic information 
source comprises a travel database. 

18. The method of claim 1, wherein the step of enabling 
the user to display a list of search terms enables the user to 
pass the pointing device over any of the displayed category 
names in a non-sequential manner. 

19. A method for searching a database on a computer 
having a display device and a pointing device controlling a 
cursor displayed on the display device, the database asso- 
ciated with at least one of a plurality of categories of 
information, at least one category having a sub-category of 
information, the method comprising: 

displaying each category on the display device; 

enabling the user to select a category by passing the cursor 
over one of the displayed categories; 

displaying on the display device, in response to the user 
selecting a category associated with sub-categories, 
each sub-category of information associated with the 
selected category; 

enabling the user to select a sub-category by passing the 
cursor over one of the displayed sub-categories; 

displaying on the display device, in response to the user 
selecting a sub-category, a list of search terms associ- 
ated with the selected sub-category; 

enabling the user to select one of the displayed search 
terms; and, 

searching the database in response to the selected search 
term. 

20. The method of claim 19, wherein at least one list of 
search terms is updated in response to the user selecting one 
of the displayed search terms. 

21. The method of claim 19, wherein at least one list of 
search terms is dynamically generated by a search of the 
information source. 

22. A system for searching a database on a computer from 
a user terminal including a display device and a manually- 
operable input unit controlling a cursor displayed on the 
display device and a select button, the database associated 
with at least one of a plurality of categories of information, 
the system comprising: 

a computer including a database and a computer processor 
for accessing the records in accordance with search 
terms selected at the user terminal: 
means for transferring a list of categories and search 
terms to the display device for simultaneous display 
of category names associated with each of said 
categories; 

means for enabling the user to select one of the cat- 
egories by passing the cursor over one of the dis- 
played category names; 

means for displaying the search terms associated with 
the selected category; 
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means for enabling the user to select one of the dis- 
played search terms; and, 

means for searching the available records of the data- 
base for records responsive to the selected search 
5 term. 

23. The system of claim 22, wherein at least one list of 
search terms is updated in response to the user selecting one 
of the displayed search terms. 

24. The system of claim 22, wherein at least one list of 
10 search terms is dynamically generated by a search of the 

database. 

25. A website having a user interface for searching a 
database of records from a user terminal having a display 

15 device and a controller for controlling a pointing device 
displayed on the display device, the website comprising: 
category names displayed on the display device; 
means for enabling the user to select a category by passing 
the pointing device over one of the displayed catego- 
20 ries; 

a list of search terms displayed on the display device in 

response to the user selecting a category; and, 
means for enabling the user to select one of the displayed 
25 search terms. 

26. The website of claim 25, wherein the database is 
located remote from the user terminal, further comprising 
means for searching the database in response to the selected 
search term. 

30 27. The website of claim 25, wherein at least one list of 
search terms is updated in response to the user selecting one 
of the displayed search terms. 

28. The website of claim 25, wherein at least one list of 
search terms is dynamically generated by a search of the 

35 database. 

29. A method for controlling a user interface for searching 
a database from a computer having a display device and a 
controller for controlling a pointing device displayed on the 
display device, the method comprising: 

40 displaying category names on the display device; 

enabling the user to select a category by passing the 

pointing device over one of the displayed categories; 
displaying a list of search terms on the display device in 

response to the user selecting a categories; and, 

45 

enabling the user to select one of the displayed search 
terms. 

30. The method of claim 29, wherein the method is 
implemented by a website. 

5Q 31. The method of claim 29, wherein at least one list of 
search terms is updated in response to the user selecting one 
of the displayed search terms. 

32. The method of claim 29, wherein at least one list of 
search terms is dynamically generated by a search of the 

55 database. 

***** 
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ABSTRACT 



A medical records system that creates and maintains all 
patient data electronically. The system captures patient data, 
such as patient complaints, lab orders, medications, 
diagnoses, and procedures, at its source at the time of entry 
using a graphical user interface having touch screens. Using 
pen-based portable computers with wireless connections to 
a computer network, authorized healthcare providers can 
access, analyze, update and electronically annotate patient 
data even while other providers are using the same patient 
record. The system likewise permits instant, sophisticated 
analysis of patient data to identify relationships among the 
data considered. Moreover, the system includes the capabil- 
ity to access reference databases for consultation regarding 
allergies, medication interactions and practice guidelines. 
The system also includes the capability to incorporate legacy 
data, such as paper files and mainframe data, for a patient. 
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ELECTRONIC MEDICAL RECORDS 
SYSTEM 

RELATED APPLICATIONS 

This application is a continuation of U.S. application No. 
09/333,170 filed on Jun. 14, 1999 which in turn is a cont. of 
U.S. application No. 08/721,182 filed on Sep. 27, 1996. 

BACKGROUND OF THE INVENTION 

1. Field of the Invention 

The present invention relates to electronic healthcare 
systems, and more particularly, to a system for storage and 
retrieval of electronic medical records in a computer 
environment, such as a local or wide area network including 
portable computers. 

2. Description of Related Technology 

Healthcare providers, such as physicians, create large 
volumes of patient information during the course of their 
business at healthcare facilities, such as hospitals, clinics, 
laboratories and medical offices. For example, when a 
patient visits a physician for the first time, the physician 
generally creates a patient file including the patient's medi- 
cal history, current treatments, medications, insurance and 
other pertinent information. This file generally includes the 
results of patient visits, including laboratory test results, the 
physician's diagnosis, medications prescribed and treat- 
ments administered. During the course of the patient 
relationship, the physician supplements the file to update the 
patient's medical history. When the physician refers a 
patient for treatment, tests or consultation, the referred 
physician, hospital, clinic or laboratory typically creates and 
updates similar files for the patient. These files may also 
include the patient's billing, payment and scheduling 
records. 

Healthcare providers can use electronic data processing to 
automate the creation, use and maintenance of their patient 
records. For example, in U.S. Pat. No. 5,277,188, assigned 
to New England Medical Center Hospitals, Inc., Selker 
discloses a clinical information reporting system having an 
electronic database including electrocardiograph related 
patient data. Similarly, Schneiderman discloses a computer 
system for recording electrocardiograph and/or chest x-ray 
test results for a database of patients in U.S. Pat. No. 
5,099,424. In U.S. Pat. No. 4,315,309, Coli discloses a 
patient report generating system for receiving, storing and 
reporting medical test data for a patient population. Mitchell, 
in U.S. Pat. No. 3,872,448, likewise discloses a system for 
automatically handling and processing hospital data, such as 
patient information and pathological test information using 
a central processing apparatus. In U.S. Pat. No. 5,065,315, 
Garcia discloses a computerized scheduling and reporting 
system for managing information pertinent to a patient's 
stay in the hospital. However, these electronic data process- 
ing systems can not handle patient data in the wide variety 
of data formats typically produced by healthcare providers, 
such as physicians, laboratories, clinics and hospitals. 

Physicians often use paper based forms and charts to 
document their observations and diagnosis. Laboratories 
also produce patient data in numerous forms, from x-ray and 
magnetic resonance images to blood test concentrations and 
electrocardiograph data. Clinics and hospitals may use a 
combination of paper based charts and electronic data for 
patient records. The same patient data may exist in remote 
patient files located at clinics, hospitals, laboratories and 
physicians' offices. Similarly, patient files at one healthcare 
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provider typically have different information than patient 
files at another healthcare provider. When in use, patient files 
are generally not available to other healthcare providers. In 
addition, at the time of creation, patient data is generally not 

5 available for use by remotely located healthcare providers. 
Moreover, relationships among specific patient data, such as 
abnormal laboratory test results, prescribed medications to 
address the abnormality, and specific treatments adminis- 
tered by the physician, may not be apparent within a patient 

10 file. 

In the current environment, specific patient data is difficult 
to access when needed for analysis. The creation of patient 
data in remote locations exacerbates this problem. In 
addition, the wide variety of data formats for patient data 

15 hinders electronic processing and maintenance of patient 
files. Moreover, the use of a patient's file by one healthcare 
provider can preclude its simultaneous use by another 
healthcare provider. Ongoing consolidation of healthcare 
providers into large health maintenance organizations 

20 (HMOs) and preferred provider organizations (PPOs) create 
issues in the transfer and maintenance of patient data in large 
enterprises having numerous remote locations. Under these 
circumstances, healthcare providers have difficulty provid- 
ing effective treatment for their patients. 

25 SUMMARY OF THE INVENTION 

The electronic medical record (EMR) system of the 
present invention automates and simplifies existing methods 
of patient chart creation, maintenance and retrieval. In 

30 contrast to other systems, the present invention creates and 
maintains all patient data electronically and thus can elimi- 
nate or supplement creating and maintaining of physical data 
records. The EMR system furnishes healthcare providers 
with an intuitive, easy-to-use, icon-based interface that 

35 enables them to capture and analyze patient data quickly and 
efficiently. Using the present invention, healthcare providers 
enter patient data immediately at the point of care. Thus, the 
EMR system captures each piece of data at its source at the 
time of entry to provide a complete audit trail for all patient 

40 data. In this manner, the EMR system transforms a patient 
chart from a static record of a few clinical interactions into 
a dynamic, real-time comprehensive record linked to an 
enterprise-wide clinical database. In addition, the EMR 
system of the present invention includes the capability to 

45 manage a wide variety of patient data formats, including 
patient data from external sources, such as laboratories and 
pharmacies. The EMR system can also incorporate a 
patient's legacy data, such as a paper chart, into the patient 
record as well as legacy data from mainframe computers. 

so The present invention likewise provides instant access to 
a patient's electronic medical record by authorized health- 
care providers from any geographical location. Thus, the 
EMR system enables authorized healthcare providers to 
access and update patient files using wireless pen-based 

55 personal computers. To enable complete replacement of 
physical records, the present invention permits healthcare 
providers, such as physicians or nurse practitioners, to 
electronically annotate patient data. Thus, a healthcare pro- 
vider can acknowledge reviewing patient data, provide 

60 instructions, such as prescriptions for medication to admin- 
ister to a patient, and approve recommendations for treat- 
. ment by other providers, all by electronically annotating a 
patient's record. In addition, authorized healthcare providers 
can access a record while other providers use the same 

65 record allowing for real-time collaboration. The availability 
of electronic data permits instant, sophisticated analysis of 
patient data. Moreover, the EMR system enables enhanced 
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analysis of patient data by providing access to reference FIG. 8 shows an example of a viewer window displaying 

databases for diagnosis, procedures and medication. an image of patient data of the point of care system of FIG. 

One aspect of the present invention includes a medical 4. 

records system, comprising a point of care system to capture FIG. 9 is a block diagram illustrating the structure of a 

patient data at a point of care and a patient data repository, 5 medication data capture in the point of care system of FIG. 

in communication with the point of care system and with 4, 

external systems, to store and organize the patient data for F IG. 10 is a block diagram illustrating the structure of a 

access by the point of care system. practice guideline in the point of care system of FIG. 4. 

Another aspect of the present invention includes a medi- FIG. 11 is a block diagram illustrating the structure of the 

cal records system comprising a point of care system to me dication data capture and the practice guideline in the 

capture data in a patient record at a point of care, wherein the poim of care system of FIG 4 

patient record includes a patient identifier and at least one i - . , , , •« . .* *u * , r.u 

\ t . . . i j ■ t if . * ■ j i .< * . FIG. 12 is a block diagram illustrating the structure of the 

data structure including the patient identifier and the data. . t , t . % ri „ + & 

* , patient data repository of FIG. 1. 

Yet another aspect of the present invention includes a ___ ... , .„ , *• *^ * * r 

,., , f. • . r ..is FIG. 13 is a block diagram illustrating the structure of a 

medical records system comprising a point of care system to \ / V *u* *u . • *T s " r ™ i- 

Jt . ■ 4 r j .*. j. •* patient record within the patient data repository of FIG. 12. 

capture data at a point of care and a patient data repository, r r r J 

in communication with the point of care system and with. , FIG - 14 is an example of the patient record of FIG. 13. 

external systems to store and organize the data in a patient FIG. 15a is a flowchart illustrating the process flow of the 

record for access by the point of care system, wherein the patient data repository of FIG. 12. 

patient record includes a patient identifier and at least one FIG. 15b is a flowchart illustrating the process for a 

data structure including the patient identifier and the data. transfer of data from a cache to a data archive in the patient 

In addition, another aspect of the present invention data repository of FIG. 12. 

includes a method of using an electronic medical records piG. 16 is a block diagram illustrating the structure of the 

system, comprising the steps of capturing patient data elec- 25 data interface of FIG. 12. 

ironically at the point of care, organizing the patient data so FIG. 17a is a flowchart illustrating the process flow of the 

as to form a patient record, filing the patient record, and data interfaoe of pjQ 16 wnen receiving pa t ien t data from 

retrieving the patient record to access the patient data for use an externa j source 

in the care of a patient. -_..«, .„ . a r • 

, e x . • • , , FIG. 17b is a flowchart illustrating the process flow of the 

Yet another aspect of the present invention includes a 30 ^ of nG u wheQ transmittj tien , data t0 

method of retrieving patient data in an electronic medical afi externa j source 

records system having a patient data repository, comprising mrt .„ . 

the steps of obtaining a patient identifier, locating a patient 18 » a K block * a fl am Crating *e structure of the 

record corresponding to the patient identifier in the patient reference database or. FIG. 1. 

data repository, and determining the location of the patient 35 FIG. 19 shows an example of a graphical user interface of 

data within the patient record. tne P oin t of care system of FIG. 4 having a reference access 

Another aspect of the present invention includes a method button and a medication manager button, 

of managing a patient data repository having a cache and a FIG. 20 shows an example of a graphical user interface 

data archive, comprising the steps of monitoring a status of for the diagnosis module and the procedure module of the 

data within the cache, and moving the data to the data 40 reference database of FIG. 18. 

archive when the status exceeds a threshold. FIG. 21 shows an example of a graphical user interface 

Still another aspect of the present invention includes a for the medication manager of the reference database of FIG. 

method of communicating with an external source having an 18. 

interface to an electronic medical records system, compris- fig. 22 shows an example of a medication interaction 

ing the steps of finding an interface for the external source, 45 window of the medication manager of FIG. 21. 

connecting to the external source using the interface, and FIG. 23 is a block diagram illustrating the structure of the 

converting patient data for transfer between the external legacy daU system of FIG L 

source and the electronic medical records system. r^-^- t r * • 1 c *-**u 

J FIG. 24 is an example of a typical configuration for the 

BRIEF DESCRIPTION OF THE DRAWINGS 50 electronic medical records system of the present invention. 

FIG. 1 is a block diagram illustrating the electronic DETAILED DESCRIPTION OF THE 

medical record (EMR) system architecture of the present PREFERRED EMBODIMENTS' 

'ToTis a flowchart illustrating the process flow of the . foll ° win S de , tailed H description of the preferred 

EMR system of the present invention. « ^boduncnts presents a description of certain spcc.flc 

\ , r ... . embodiments to assist in understanding the claims. 

FIG. 3 shows an example of a graphical user interface of However> one may p r actice the present invention in a 

the EMR system useful for the scheduling of a patient multitude of different embodiments as defined and covered 

appointment as shown in FIG. 2. by the daims 

FIG 4 is a block diagram illustrating the structure of the 6Q For conveniencej me description comprises three sec- 
point of care system of FIG. 1. tions: EMR System Architecture and Overview, EMR Sys- 

FIG. 5 shows an example of a graphical user interface of t em Configurations and Summary. The first section provides 

the point of care system of FIG. 4. an overview of the EMR system architecture, the following 

FIG, 6 shows an example of a new form window of the . section describes EMR system applications and preferred 

point of care system of FIG. 4. 65 embodiments for practicing the EMR system of the present 

FIG. 7 shows an example of an annotate window of the invention, and the remaining section summarizes advanta- 

point of care system of FIG. 4. geous features of the present invention. 
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I. EMR System Architecture and Overview 

FIG. 1 illustrates the architecture of the EMR system. 
Healthcare providers, such as physicians, at hospitals, labo- 
ratories and clinics, generally capture and access patient data 
using a point of care system 100 that communicates with a 
patient data repository 102. Patient data, such as vital signs, 
x-ray images and laboratory results, resides in the patient 
data repository 102. The patient data repository 102 also 
communicates with external sources to obtain patient data, 
such as laboratory test results and x-ray images, and to 
transfer patient information, such as prescriptions for 
medication, from the EMR system to other healthcare pro- 
viders. The point of care system 100 captures patient data in 
realtime at the point of care, that is, where healthcare 
providers interact with their patients. For example, physi- 
cians can use a point of care system 100 to enter, access, 
process, analyze and annotate data from patient records in 
real-time at the point of care. Thus, using the point of care 
system 100, a physician, who has many patients in a 
hospital, can visit each patient in their room, access their 
electronic patient record there, enter results of the current 
examination, evaluate their medical history, electronically 
annotate their x-rays images and prescribe medications and 
treatments instantaneously as the point of care system 100 
captures and organizes patient data into the patient record 
stored in the patient data repository 102. The point of care 
system 100 may likewise communicate with a reference 
database 104 to assist a healthcare provider in making 
diagnoses, prescribing medications and administering treat- 
ments. Moreover, the patient data repository 102 may also 
communicate with a legacy data system 106 to access 
pertinent patient data in paper files and mainframe electronic 
databases. 

Referring now to FIG. 2, a flowchart illustrates the 
operation of the EMR system. For example, a patient having 
a complaint contacts a healthcare provider 110, such as a 
physician, to schedule an appointment. The EMR system 
obtains the patient record 111 from the patient data reposi- 
tory 102 (FIG. 1) prior to the scheduled appointment. The 
EMR system is also capable of handling patients on a 
walk-in basis by scheduling an appointment and requesting 
the patient's record immediately thereafter. The EMR sys- 
tem updates the patient record 112 to include the complaint 
and other information pertinent to the appointment, such as 
insurance information. A healthcare provider, such as a 
physician, examines the patient 113 using the point of care 
system 100 (FIG. 1) to make a diagnosis and to treat the 
patient's condition. As determined at 114, if a diagnosis is 
not possible on the basis of this examination, the physician 
may need to obtain additional clinical data 115, such as 
laboratory tests and x-rays. When available, the physician 
uses the point of care system 100 (FIG. 1) to evaluate the 
results 116 and to examine the patient 113 again in light of 
the results. Upon making a diagnosis, the physician may 
need to prescribe medications 117 for the patient's condi- 
tion. Similarly, the physician may need to administer a 
treatment 118 to address the patient's condition. At the 
conclusion of the patient's visit, the EMR system files the 
patient's record 119 in the patient data repository 102 (FIG. 
1) for future reference. 

In a preferred embodiment, the EMR system includes 
graphical user interfaces to access system functions. For 
example, as shown in FIG. 3, a chart puller window 120 
enables a healthcare provider to schedule a patient appoint- 
ment using its point and click interface. To schedule an 
appointment, a healthcare provider activates the select but- 
ton 121 with a pointing device, such as a mouse or electronic 
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pen, to obtain a list of patients. The healthcare provider then 
scans the list to select the name of the appropriate patient 
using a pointing device. The EMR system places the name 
of the selected patient in the patient box 123. Similarly, the 

5 healthcare provider uses the up/down buttons 125 to select 
an appointment date and an appointment time. An adjacent 
box, such as the date box 126, displays the selected date and 
time. Lastly, the healthcare provider enters a textual descrip- 
tion of the patient's complaint in a reason box 127. Note that 

10 the healthcare provider can review prior or future scheduled 
appointments by clicking on the appointments button 128. 
Similarly, the healthcare provider can track referrals by 
entering the identity of persons who referred this patient to 
their care in the referral box 129. 

15 Referring now to FIG. 4, a block diagram illustrates the 
structure of the point of care system 100. The point of care 
system 100 includes the following modules: a patient data 
capture 140, a clinical data capture 142, progress notes 144 
and an encounter data capture 146. During a patient visit, the 

20 healthcare provider (not shown) can enter, review and anno- 
tate patient information, such as family history, 
appointments, current medications and complaints, using the 
patient data capture 140. The healthcare provider can like- 
wise enter, review and annotate clinical data obtained during 

25 the visit, such as body temperature and blood pressure, using 
the clinical data capture 142. Similarly, the healthcare pro- 
vider can enter laboratory data for patients with the clinical 
data capture 142. The clinical data capture 142 communi- 
cates with the patient data capture 140 to assist in identifying 

30 needs for further clinical data. For example, a family history 
of high blood pressure may indicate a need to obtain the 
patient's blood pressure during the visit. The patient data 
capture 140 also communicates with the encounter data 
capture 146, where a healthcare provider can enter, review 

35 and annotate data regarding diagnoses and procedures 
administered to the patient. Moreover, the healthcare pro- 
vider can use the progress notes 144 to summarize details of 
the patient's condition and to review the patient's progress 
over time. Thus, the progress notes 144 communicates with 

40 the patient data capture 140, the clinical data capture 142 
and the encounter data capture 146. 

Referring now to FIG. 5, in a preferred embodiment, the 
point of care system 100 (FIG. 1) includes a graphical user 
interface having a patient chart window 150 to capture 

45 patient information. The point of care system 100 presents a 
patient record graphically using a tabbed layout to organize 
patient data. The patient chart window 150 includes tabs for 
patient data 151, clinical data 152, encounter data 153 and 
progress notes 154. Pointing and clicking on a tab on the 

50 patient chart window 150 opens a folder window 155 where 
a healthcare provider can enter and review patient data 
within the folder. For example, to activate progress notes 
144 (FIG. 4), the healthcare provider selects the progress 
notes tab 154 to display a list of progress note data in the 

55 folder window 155. In a similar manner, to activate the 
patient data capture 140, the clinical data capture 142 or the 
encounter data capture 146, one selects the patient data tab 
151, the clinical data tab 142, or the encounter data tab 153, 
respectively. 

60 To enter patient data, the healthcare provider clicks on the * 
scroll down button 156 to select a form from a list of 
available forms to enter patient data. This activates the new 
forms box 157. The provider then points and clicks on the 
new form button 158. For example, FIG. 6 shows a new 

65 form window 161 displaying the pediatric problem form 162 
selected by the healthcare provider using the scroll down 
button 156 (FIG, 5). The healthcare provider fills out the 
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pediatric problem form 162 using an input device, such as a 
keyboard, a mouse or an electronic pen. For example, the 
provider uses a keyboard to enter text "Jun. 7, 1996 Stomach 
Ache" 164 and an electronic pen to enter initials 166 for 
identification. When done with patient data entry, the pro- 
vider exits the form using the File Menu 168 and the point 
of care system 100 returns the provider to the patient chart 
window 150 (FIG. 5). Referring back to FIG. 5, the new 
form appears as the top entry of the list in the folder window 
155. 

Similarly, to annotate patient data, the healthcare provider 
first selects an item to annotate by pointing and clicking on 
the item in a list displayed in the folder window 155. The 
provider then clicks on the annotate button 159 to open the 



with a historical context of the patient's condition and 
alternative treatments already attempted. 

FIG. 11 shows a point of care system 100 having a 
medication data capture 148 and a practice guideline 149. As 
5 before, the medication data capture 148 communicates with 
the patient data capture 140 and with the progress note 144. 
Similarly, the practice guideline 149 communicates with 
patient data capture 140, the clinical data capture 142, the 
encounter data capture 146 and the progress note 144. 
10 However, the practice guideline 149 may now communicate 
with the medication data capture 148 to address situations 
where accepted practice guidelines require a healthcare 
provider to prescribe and administer medications. In a 
preferred embodiment, the point of care system 100 includes 
item in an annotate window 170, as shown in FIG. 7. For is the graphical user interface illustrated in FIG. 5. Referring 
example, the annotate window 170 of FIG. 7 displays a back to FIG. 5, the patient chart window 150 includes tabs 
blood test result 172. As before, the healthcare provider for medication data 191 and practice guidelines 193 that 
annotates the blood test result document 172 using an input activate the medication data capture 148 and the practice 
device, such as a keyboard, a mouse or an electronic pen. For guideline 149, respectively. Similarly, pressing the medica- 
example, the provider uses a keyboard to enter text "Out of 20 tion manager button 192 activates the medication data 
Range" 174 and an electronic pen to circle 176 the out of capture 148 and the practice guideline 149. A healthcare 
range result. When done with annotations, the provider exits provider can enter, review and annotate patient medication 
the form using the File Menu 178 and the point of care data and practice guideline data as described previously, 
system 100 returns the provider to the patient chart window Referring now to FIG. 12, a block diagram illustrates the 
150 (FIG. 5). Note that the point of care system 100 tracks 25 structure of the patient data repository 102. The patient data 
the review of patient data and identifies reviewed files with repository 102 includes a patient locator 200, a data manager 
a mark 160 in the folder window 155. By annotating patient 202 and a data interface 204. The patient locator 200 
data, a healthcare provider, such as a physician, can generates a unique patient identifier (PID) 221 (FIG. 14) for 
acknowledge reviewing patient data, provide instructions, each patient and creates and maintains a table having PIDs 
such as directions for additional tests and procedures or 30 for all patients who have data in the patient data repository 
prescriptions for medication to administer to the patient, and 102, All data records related to a patient 211, 212, 213, 214, 
approve recommendations for treatment by other healthcare 215, 216, 219 include and reference the patient's unique PID 
providers. Lastly, as shown in FIG. 8, a healthcare provider as shown in FIG. 13. 

uses the patient chart window 180 to view patient data. First, With reference to FIG. 13, upon creation of a patient 
the healthcare provider selects a view item 182 by either 35 record, the patient locator 200 creates a patient data structure 
pointing and clicking twice on the item in a list displayed in 210 having the PID and the patient's name. In a preferred 
the folder window 184 or by pointing at the item in the list embodiment, the patient data structure 210 includes pointers 
and pressing the view button 183. The double click opens a to data structures having data within a patient record cap- 
viewer window 185 to display the view item 182. For tured by the point of care system 100 and incorporated from 
example, the viewer window 185 of FIG. 8 displays an x-ray 40 external sources (e.g., a digital x-ray image file stored in a 



186. As before, the healthcare provider may annotate the 
x-ray 186 with comments and observations by clicking on 
the annotate button 187. The healthcare provider may like- 
wise close the viewer window 185 by clicking on the close 
button 189. 

Certain additional structures in the point of care system 
100 (FIG. 1) will now be discussed with reference to FIGS. 
9, 10 and 11. Referring now to FIG. 9, an optional medica- 
tion data capture 148 supplements the structure of the point 



raster pixel format). Thus, the patient data structure 210 
maintains a pointer to an interface files structure 211 having 
patient data transmitted from external sources. The patient 
data structure 210 likewise maintains pointers to a clinical 
45 data structure 212, a progress note structure 213 and an 
encounter data structure 214. These data structures include 
patient data captured by the clinical data capture 142, 
progress notes 144 and encounter data capture 146, respec- 
tively (FIG. 4). In another preferred embodiment, the patient 
of care system 100 of FIG. 4. A medication data capture 148 50 data structure 210 may include pointers to data structures 
allows a healthcare provider to monitor a patient's medica- having data generated by the reference database 104 and 
tions. The medication data capture 148 communicates with transferred by the legacy data system 106. Thus, the patient 
the patient data capture 140 to account for medications the data structure 210 may maintain pointers to a medication 
patient is currently taking. The medication data capture 148 data structure 215 and a guideline data structure 216. As 
similarly communicates with the progress notes 144, where 55 described above, the medication 215 and guideline 216 data 
a practitioner can monitor changes in a patient's condition structures include patient data captured by the medication 
resulting from medication therapies. Referring now to FIG. data capture 148 and the practice guideline 149, respec- 
10, an optional practice guideline 149 supplements the tively. In this embodiment, a reference data structure 217 
structure of the point of care system of FIG. 4. The practice may maintain pointers to the encounter data structure 214 
guideline 149 provides references for practitioners to consult 60 and to the medication data structure 215 for access to 
regarding courses of action to obtain a diagnosis and alter- reference information contained in a reference database 104. 
native treatments for various conditions. The practice guide- Lastly, the patient data structure 210 may maintain a pointer 
line 149 communicates with the patient data capture 140, the to a legacy files structure 219 having patient data transmitted 
clinical data capture 142 and the encounter data capture 146 from the legacy data system 106, such as an image of a 
to assist the practitioner in selecting the appropriate course 65 patient chart, 

of action. The practice guideline 149 likewise communicates FIG. 14 shows a logical view of a patient record 220 
with the progress notes 144 to provide a healthcare provider corresponding to the structure illustrated in FIG. 13. The 
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patient record 220 includes the PID generated by the patient 
locator 200 (FIG. 12) in the patient data repository 102 (FIG. 
1). In addition, the patient record 220 includes patient data 
in a variety of data types generated by healthcare providers. 
Thus, the patient record includes text data 223, such as 
electronic mail and word processing documents from other 
healthcare providers, image data 225, such as scanned 
physical documents, x-rays and CATSCANs, and audio data 
227, such as a physician's dictation and voice mail. Lastly, 
the patient record 220 has data tables 229, such as a 
physician's ICD9 diagnosis codes and CPT procedure 
codes. In view of the structure of a patient record 220, 
referring back to FIG. 12, the data manager 202 uses the PID 
to store and retrieve patient records. Moreover, the data 
interface 204 permits communication with external sources 
to obtain patient data, such as demographic data, laboratory 
test results and x-ray images, and to transfer patient 
information, such as prescriptions for medication, from the 
patient data repository 102 to external healthcare providers. 

With reference to FIG. 12, the patient data repository 102 
may optionally include a cache 206 for temporary storage of 
patient data and a data archive 208 for long term storage of 
patient data. In this embodiment, the data manager 202 
coordinates the transfer of patient data to and from a data 
archive 208 into a cache 206. For example, the data manager 
202 may identify patient records that a healthcare provider 
needs for appointments scheduled at a future time and then 
transfer these patient records from the data archive 208 into 
the cache 206 for quick access prior to the scheduled 
appointment. Similarly, the data manager 202 may purge 
from the cache 206 records of patients who have not had 
recent appointments and whose records are already 
archived. The data manager 202 likewise tracks the location 
and description of patient data within the data archive 208 by 
associating the file name of the patient data within a patient 
record 220 with the patient identifier 221. When possible, 
the data manager 202 will group data associated with a 
patient within the data archive 208 for rapid retrieval in a 
manner similar to files within a directory in an operating 
system. Thus, the data manager 202 assigns a directory to 
each patient identifier and then stores patient data within this 
directory. 

FIG. 15a illustrates the process flow for the patient data 
repository 102 (FIG. 1). For example, the point of care 
system 100 (FIG. 1) issues a request for patient data 250. 
With reference to FIGS. 15a and 12, the patient locator 200 
receives the request from the point of care system 100 and, 
at 251 attempts to find the PID for the record having the 
requested patient data. As determined at 252, if no PID is 
found, the patient locator 200 reports an error 253. At this 
point, the patient data repository 102 (FIG, 1) may recover 
from the error 253 by either restarting the process or by 
ending the process. Otherwise, the patient locator 200 com- 
municates the PID to the data manager 202. The data 
manager 202 locates the patient record using the PID at 254. 
As determined at 255, in a system without cache 206 and 
without a data archive 208, the data manager 202 delivers 
the requested data 256 to the point of care system 100. In a 
system having a cache 206 and a data archive 208, the data 
manager 202 determines at 257 if the requested data exists 
in the cache 206. If so, the data manager 202 delivers the 
requested data 256 to the requester from the cache 206. 
Otherwise, the data manager 202 first moves the data 258 
from the data archive 208 to the cache 206 and then delivers 
the requested data 254 to the requester from the cache 206. 

In addition, FIG. 15b, in conjunction with FIG. 12, 
illustrates the process for transferring data from a cache 206 
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to a data archive 208. The data manager 202 monitors the 
contents of the cache 206. To improve the performance of 
the cache 206, the data manager 202 requests transfer 260 of 
data to the data archive 208 under certain conditions. For 
example, the data manager 202 may purge the cache 206 
when data requested for storage in the cache would exceed 
its memory capacity. In this circumstance, the data manager 
202 first transfers to the data archive 208 signed files and 
then data files in chronological order, i.e., oldest files first. 
Similarly, a healthcare provider can specify a predetermined 
time, such as 3 calendar days, or other selected conditions 
for transfer to the data archive 208. As determined at 262, if 
the cache 206 does not have the data to transfer, the process 
ends as the data manager 202 ignores the request. As 
determined at 264, if the data in the cache 206 is not ready 
for transfer, the process ends and the data manager 202 
queues the request for the next transfer of data to the data 
archive 208. Data in the cache 206 is ready for transfer when 
a physician has reviewed and accepted it and when it has not 
been previously committed to the data archive 208. 
Otherwise, the data manager 202 transfers data from the 
cache 206 to the data archive 208 at 266. 

Referring now to FIG. 16, the data interface 204 of the 
patient data repository 102 includes an interface manager 
270, a data handler 272 and a communication interface 274. 
To transfer and receive patient data from external sources 
(not shown), the interface manager 270 communicates with 
a data handler 272 and a communication interface 274. In 
addition, the communication interface 274 communicates 
with the data handler 272 for conversion of received external 
patient data into formats recognized by the EMR system. 
The interface manager 270 creates and maintains an inter- 
face registry of data formats for external sources. Prior to 
data transfer or receipt by the EMR system, the interface 
manager 270 registers an interface for an external source. 
Upon registration of an interface, the interface manager 270 
can provide the appropriate conversion routines for the data 
handler 272 to use for transfer of data to and receipt of data 
from an external source. These conversions are well under- 
stood by the relevant technologist. 

FIGS. 17a and 17b illustrate the operation of the data 
interface 204 of the patient data repository 102 (FIG. 12). 
Referring now to FIG. 17a, the data manager 202 issues a 
request 280 for patient data from an external source. At 282, 
the interface manager 270 determines if the registry includes 
an interface for the external source, such as a laboratory or 
pharmacy. As determined at 282, if the registry includes an 
interface for the external source, the communication inter- 
face 274 connects to the external source 284 to receive 
patient data. The data handler 272 retrieves the appropriate 
conversion routine for the external source to convert data 
286. In a preferred embodiment, the data handler 272 
converts data from an external source into a database table 
for the appropriate PID. Lastly, the data manager 202 
incorporates converted data 288 into the patient record. 
Otherwise, the interface manager 270 reports an error 289. 
The data manager 202 may recover from the error 289 in 
several ways. First, the data manager 202 may invoke a 
module to register an interface for the external source so as 
to allow the process to continue. Second, the data manager 
202 may end the process at this point. Lastly, the data 
manager 202 may restart the process in the event the external 
source was specified incorrectly. 

Referring now to FIG. 17 b, an external source requests 
data 290 from a patient record. As described above, the 
interface manager 270 determines at 292 if the registry 
includes an interface for the external source. As determined 
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at 292, if the registry includes an interface for the external 
source, the data manager 202 locates the requested data at 
294 and the data handler 272 converts requested data at 296 
to the format required by the external source. The commu- 
nication interface 274 then sends the converted data to the 
external source at 298. For example, the patient data reposi- 
tory 102 may transmit a physician's prescription for medi- 
cation to a hospital or pharmacy. If the registry includes no 
interface for the external source, the interface manager 270 
reports an error 299. Similarly, as discussed above for the 
process flow of FIG. 17a, the interface manager 270 may 
recover from the error 299 by restarting the process, ending 
the process or invoking a module to register the external 
source to allow the process to continue. 

Referring now to FIG. 18, a block diagram illustrates the 
structure of the optional reference database 104 (FIG. 1), 
The reference database 104 includes a diagnosis module 
300, a medication manager 302 and a procedure module 
304. A healthcare provider can use the reference database 
104 for assistance in diagnosing a patient's disease, pre- 
scribing medications and ordering supplemental procedures 
to treat the disease. The diagnosis module 300 communi- 
cates with a medication manager 302 to obtain information 
on medications indicated by a diagnosis. The medication 
manager 302 provides information on medications, such as 
proper dosages, allergies, contraindications, adverse inter- 
actions with other medications, and side effects. The diag- 
nosis module 300 likewise communicates with a procedure 
module 304 to obtain information on the proper adminis- 
tration of procedures indicated by a diagnosis. The proce- 
dure module 304 provides information on procedures for 
treatment as indicated by the diagnosis. In many instances, 
the medication manager 302 communicates with the proce- 
dure module 304 regarding the administration of- various 
medications. 

In a preferred embodiment, the point of care system 100 
provides access to the reference database 104 through a 
graphical user interface having a patient chart window 310 
shown in FIG. 19. A healthcare provider accesses the 
diagnosis module 300 and the procedure module 304 by 
pointing and clicking on a reference access button 312. 

As shown in FIG. 20, the reference access button 312 
produces a reference window 330 including the graphical 
interfaces for the diagnosis module 300 and the procedure 
module 304. For example, to enter a diagnosis, a physician 
clicks on the scroll down button 331 adj acent to the system 
box 332 to produce a list of body systems. The physician 
selects the appropriate system and the diagnosis module 300 
enters the selected system in the system box 332 and 
provides a list having specific diagnosis codes for the 
selected body system in the diagnosis box 334. The physi- 
cian then selects the appropriate diagnosis code and clicks 
on the add button 336 adjacent to the diagnosis selection box 
337. The diagnosis module 300 enters the selected diagnosis 
code to the diagnosis selection box 337. The physician may 
repeat the above steps to add multiple diagnosis codes to the 
diagnosis selection box 337. In a similar manner, a physician 
uses the scroll down button 331 adjacent to the topic box 333 
to select the appropriate procedure topic. The procedure 
module 304 enters the selected procedure topic in the topic 
box 333 and provides a list of procedure codes in the 
procedure box 335. The physician now selects the appro- 
priate procedure code and adds it to the procedure selection 
box 338 by clicking on the add button 336 adjacent to the 
procedure selection box 338. The physician may likewise 
repeat the above steps to add multiple procedure codes to the 
procedure selection box 338. The physician completes entry 
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of diagnoses and procedures by clicking on the done button 
339 to return to the patient chart window 310 of FIG. 19. 

The healthcare provider similarly accesses the medication 
manager 302 (FIG. 18) by clicking on a medication button 

5 192 (FIG. 19). Referring now to FIG. 21, the medication 
button 314 activates a medication manager window 350. The 
physician can review the patient's history by viewing the 
medication history box 351 and the diagnosis history box 
352 before prescribing any new medications. The physician 

10 can also review any patient allergies in the allergy box 353. 
The physician can select a medication by entering the name 
of the medication in the name box 354. Note that as the 
physician enters the root letters of a medication name, a list 
of medications with the root letters appears in the medica- 
ls tion list box 355. As before, the physician selects a medi- 
cation from the list by clicking on it and the medication 
manager 302 places the selected medication in a selection 
box 356. If there are no contraindications or allergies for the 
patient, the physician prescribes the medications listed in the 

20 selection box 356 by clicking on the prescribe button 357. 
Otherwise, if a contraindication exists, a warning appears 
in a warning bar 358 to alert the physician. In view of the 
warning, the physician can investigate the effects of the 
medication by clicking on the results button 359. Referring 

25 now to FIG, 22, the results button produces a medication 
interaction window 361. A medication selection box 362 
displays the medications selected and under consideration 
by the physician. An allergy list box 363 displays the 
patient's allergens. Folder tabs 364 include labels describing 

30 the medication combinations and interactions. The physician 
clicks on one of these folder tabs 364 to display the contents 
of the folder in the viewing box 365. The physician can then 
evaluate the information on the interaction including poten- 
tial adverse patient reactions. The physician clicks on the 

35 done button 366 to return to the medication manager win- 
dow 350 of FIG. 21. The physician can make any needed 
revisions to the medications selected in the manner 
described above. Afterwards, the physician exits the medi- 
cation manager 302 by clicking on the exit button 360. 

40 Referring now to FIG. 23, a block diagram illustrates the 
structure of the optional legacy data system 106 as shown in 
FIG. 1. The legacy data system 106 includes a data source 
370 and a converter 372. The data source 370 comprises 
physical data 374, such as paper based records and 

45 photographs, and electronic mainframe data 376. The con- 
verter 372 receives information from the data source 370 
and transforms the information into an electronic format 
compatible with the EMR system. For example, to input 
physical data 374, such as paper or image based data, into a 

50 patient record, the converter 372 comprises a scanner to 
digitize the physical data into a binary file format for 
incorporation into the patient's record. To input electronic 
mainframe data 376, the converter 372 employs the same 
mechanism used for transfer or receipt of patient data from 

55 external sources. As described before, the converter 372 
determines if an interface exists for the mainframe data, 
selects the appropriate data handler and converts the data 
into the proper format for incorporation into a patient record. 
II. EMR System Configurations 

60 FIG. 24 illustrates one possible configuration for the EMR 
system of the present invention. The system comprises a 
wide area network (WAN) 402, the World Wide Web (Web) 
404 portion of the Internet, and remote web servers 406, 
408, 410 communicating with web browsers 412. The WAN 

65 402 comprises a plurality of local area network (LAN) 
servers supporting local and remotely located healthcare 
providers. For example, the WAN 402 includes LANs sup- 
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porting Scripps Health 414 and Sharp Memorial 430 in San 
Diego and Cedars Sinai 432 and Loma Linda 434 in Los 
Angeles, Calif. In one presently preferred embodiment, the 
server comprises a multi-processor personal computer hav- 
ing Intel Pentium processors, such as a Compaq Proliant 
4500R 5/100 Model 2, communicating with a fault tolerant, 
error correcting storage device, such as a Hewlett Packard 
20XT Optical Jukebox having 20 gigabytes of storage 
capacity. The LAN 400 includes a backup server 426 and 
several peripherals, such as a scanner 424 to input docu- 
ments and a laser printer 422 to print out documents. In a 
preferred embodiment, the LAN backbone comprises an 
Ethernet twisted pair cable configured in a general star 
topology. Similarly, the scanner 424 comprises a Fujitsu 



specifies the semantics of various components of database 
management systems (DBMS). In particular, it defines the 
structures and operations of a data model implemented by 
the DBMS, as well as other components that support data 
5 definition, data access, security, programming language 
interface and data administration. The SQL- 92 standard 
specifies data definition, data manipulation, and other asso- 
ciated facilities of a DBMS that supports the relational data 
model. SQL is old in the art and additional information on 
10 SQL-92 is available in ANSI specification X3.135-1992, 
hereby incorporated by reference. 

Similarly, in another preferred embodiment, relational 
databases in the EMR system support the Open Database 
Connectivity (ODBC) model. ODBC is an application pro- 



M3093EX scanner using Kofax KIPP ImageControls soft- 15 gram interface (API) that allows client applications running 



ware and the laser printer 422 comprises a Hewlett Packard 
LaserJet 4Plus. Healthcare providers may access the LAN 
400 using a desktop computer 416, a laptop computer 418 or 
wireless pen computer 420. In a preferred embodiment, the 



under Microsoft Windows to access data from a variety of 
data sources, including relational and non-relational DBMS. 
These data sources may reside on a client machine or they 
may be located on a remote server communicating through 



desktop computer 416 comprises a Compaq Deskpro 5/75 20 a network common to the client machine. Under ODBC, 



Model 630, the laptop computer 418 comprises a IBM 
ThinkPad 760CD and the pen computer 420 comprises a 
Fujitsu Stylist 1000 configured with a Solectek AirLAN 
PCMCIA network adapter for wireless LAN access. The 



data sources may vary in complexity from shrink-wrap 
databases, such as Microsoft Access, running under Win- 
dows on a client machine to more sophisticated, proprietary 
relational DBMS running on a Unix server or mainframe 



EMR system also provides for communication through the 25 computer. For a client application to access data from a data 



World Wide Web. For example, remote healthcare providers 
may access the WAN 402 on the Web using the domain 
name "www.westcst.com" 436. Thus, a healthcare provider 
located in Boston, Mass. may access a patient record resi- 
dent on the Scripps Health server 414, located in San Diego, 
Calif., using a web browser 412, such as Microsoft Explorer 
or Netscape Navigator, communicating with a Web server in 
Boston, Mass. having the domain name "www.boston.com" 
406. 



source, a dynamic link library (DLL) driver must exist for 
each data source to be accessed. For additional information 
on ODBC is available from Inside ODBC, by Karl Geiger, 
hereby incorporated by reference. 
30 II. Summary 

The electronic medical record system of the present 
invention advantageously overcomes several limitations of 
existing technologies and alternatives. Because it is more 
efficient and cost effective to move data, instead of physical 



In a preferred embodiment, servers 414, 426, desktop 416, 35 records and healthcare providers, the present invention 

or laptop 418 computers and peripherals, such as printers eliminates the need to create and maintain any physical data 

422 or scanners 424, communicate with each other and with records. In contrast to other systems, the present invention 

the Web using a network operating system, such as creates and maintains all patient data electronically. Thus, 

Microsoft Windows NT, Windows 95 or Windows for Work- there is no need to find, pull, move, update, file and replace 

groups. Similarly, pen computers 420 use the Microsoft 40 physical charts. As a result, healthcare providers no longer 



Windows for Pen Computing operating system. In another 
preferred embodiment, the servers, computers and periph- 
erals communicate using an operating system supporting 
Web browsers on computer networks, such as Unix, Novell 
Netware or Apple System 7.0. In yet another preferred 
embodiment, the EMR system includes servers, computers 
and peripherals networked using mixed network operating 
systems, such as Unix, Netware and Windows. For example, 
the LAN 400 may operate on a Windows NT network 



require substantial shelving and storage space for physical 
files. The present invention likewise eliminates the 
mishandling, loss and destruction of patient data typically 
associated with maintenance of physical data records. 
45 Using the present invention, healthcare providers enter 
patient data immediately at the point of care. Thus, the EMR 
system captures each piece of data at its source at the time 
of entry, including time and healthcare provider identifica- 
tion. The EMR system thus provides a complete audit trail 



operating system, whereas the LAN 430 may operate on an 50 for all patient data. The audit trail, in turn, permits inexpen- 
Apple System 7.0 network, and the Web server "www.bos- sive analysis of outcomes, utilization and compliance. For 
ton.com" 406 may operate on a Unix operating system. example, outcomes typically refer to the effectiveness of a 
Thus, the EMR system supports communication among a treatment plan. Thus, the EMR system enables a healthcare 
variety of hardware components, such as printers 422, provider to analyze patient recovery times and incurred costs 
scanners 424 and pen computers 420, using a variety of 55 to measure the efficacy of the treatment plan. Similarly, 
network operating systems, such as Windows, Netware or utilization typically refers to how well available resources 

are utilizing time. Thus, the EMR system provides the 
capability to analyze utilization of physicians, nurses, staff 
and equipment as well as time utilization for patients, such 
60 as wait times for referrals, lab results and physician exami- 
nations. Lastly, compliance typically refers to conformance 
with government and accreditation standards and regula- 
tions. The EMR system provides tools to enable healthcare 
providers to measure conformance to standards and regula- 
forms to American National Standards Institute (ANSI) 65 tions. To facilitate entry of patient data at the point of care, 
standard SQL-92, a 580 page specification for the SQL the invention provides touch screens for entry of lab orders, 
relational database language. A database language standard medications, diagnoses and procedures. The invention like- 



Unix. In a preferred embodiment, healthcare providers, such 
as clinics and laboratories, may also communicate with the 
EMR system using modem links and standard v0.34 modem 
devices, such as a US Robotics Sportster 28,800 modem. 

The EMR system includes several databases of electronic 
information, such as the medication manager 302 and the 
data manager 202. In a preferred embodiment, the EMR 
system implements a relational database language that con- 
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wise provides instant access to a patient's electronic medical 
record by authorized healthcare providers from any geo- 
graphical location. Thus, the EMR system enables autho- 
rized healthcare providers to access and update patient files 
using wireless pen-based personal computers. In addition, 
authorized healthcare providers can access a record while 
other healthcare providers use the same record. By provid- 
ing simultaneous access to patient data, the present inven- 
tion enables real-time collaboration among multiple health- 
care providers. 

The availability of electronic data permits instant, sophis- 
ticated analysis of a patient's clinical data. Thus, the EMR 
system can create graphs of a patient's vital signs and lab 
results or the system can provide an analyze patient infor- 
mation to identify medication interactions and allergies. 
Using the present invention, a healthcare provider can 
likewise select, sort, and analyze patient data to identify 
relationships among the data considered. In addition, the 
EMR system provides flexibility in the creation and main- 
tenance of patient data repositories. Thus, the present inven- 
tion can support a large healthcare enterprise distributed 
across a large geography as well as a single physician office. 
Moreover, the present invention ensures patient confidenti- 
ality through the use of a tiered password system. The EMR 
system provides several levels of security for access to 
patient data. For example, a system administrator may have 
global password access to any patient data for system 
maintenance and debug purposes, whereas physicians may 
have access only to patient records within their specialty and 
nurses and staff may have access to only those patient 
records within their immediate care. In addition, a patient 
may request restricted access to their data by only certain 
personnel. Thus, in contrast to physical records, the EMR 
system provides superior protection of patient data. 

In addition, the present invention is useful in legal, 
manufacturing and general administration environments. 
For example, the present invention is capable of organizing, 
maintaining and protecting legal files in an attorney's office. 
Thus, the EMR system can store and retrieve scanned 
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images of paper documents, such as deeds and assignments, 
as well as other native file formats, such as word processing 
files. The EMR system organizes and retrieves this data in a 
manner akin to that of a patient's medical record. Upon entry 

5 of a client data into the EMR system, attorneys can annotate 
documents, transfer information to and from other systems, 
or create new data for automatic filing in the client or case 
file. Similarly, the EMR system is useful for management of 
procurement or regulatory data in a manufacturing context. 

io Thus, the EMR system can organize and maintain material 
safety data sheets (MSDS) as well as other data pertinent to 
materials procurement, such as conformance to specification 
measurements and inspection data for received lots, in a 
manufacturing environment. Lastly, the EMR system is 

1 5 useful for general administrative files in any organization. 
For example, the present invention is applicable to employee 
files in human resources, customer files in sales and 
approved suppliers in procurement. The EMR system can 
organize and retrieve data within these files in the manner as 

20 patient data in a patient data record. As discussed above, 
upon entry of a data into the EMR system, users can annotate 
documents, transfer information to and from other systems, 
or create new data for automatic filing in the respective file. 
Those skilled in the art may practice the .principles of the 

25 present invention in other specific forms without departing 
from its spirit or essential characteristics. Accordingly, the 
disclosed embodiments of the invention are merely illustra- 
tive and do not serve to limit the scope of the invention set 
forth in the following claims. 

30 What is claimed is: 

1. A medical records system, comprising: 
a point of care system to capture patient data at a point of 
care; and 

a patient data repository, in communication with the point 
35 of care system and with external systems, to store and 
organize the patient data for access by the point of care 
system. 

* * * * * 
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